This document is the summary of the R for Cognitive Psychologists workshop.

All correspondence related to this document should be addressed to:

Omid Ghasemi (Macquarie University, Sydney, NSW, 2109, AUSTRALIA)

Email:

1 Research Question

The aim of the study is to test if simple arguments are more effective in belief revision than more complex arguments. To that end, we present participants with an imaginary scenario (two alien creatures on a planet) and a theory (one creature is predator and the other one is prey) and we ask them to rate the likelihood truth of the theory based on a simple fact (We adapted this method from Gregg et al.,2017; see the original study here). Then, in a between-subject manipulation, participants will be presented with either 6 simple arguments (Modus Ponens conditionals) or 6 more complex arguments (Modus Tollens conditionals), and they will be asked to rate the likelihood truth of the initial theory on 7 stages.

The first stage is the base rating stage. The next three stages include supportive arguments of the theory and the last three arguments include disproving arguments of the theory. We hypothesized that the group with simple arguments shows better persuasion (as it reflects in higher ratings for the supportive arguments) and better dissuasion (as it reflects in lower ratings for the opposing arguments).

In the last part of the study, participants will be asked to answer several cognitive capacity/style measures including CRT, AOT-E, mindware, and numeracy scales. We hypothesized that cognitive ability, cognitive style, and open-mindedness are positive predictors of persuasion and dissuasion. These associations should be more pronounced for participants in the group with complex arguments because the ability and willingness to engage in deliberative thinking may favor participants to assess the underlying logical structure of those arguments. However, for participants in the simple group, the logical structure of arguments is more evident, so participants with lower ability can still assess the logical status of those arguments.

Thus, our hypotheses for this experiment are as follows:

  • Participants in the group with simple arguments have higher ratings for supportive arguments (They are more easily persuaded than those in the group with complex arguments).

  • Participants in the group with simple arguments have lower ratings for opposing arguments (They are more easily dissuaded than those in the group with complex arguments).

  • There are significant associations between CRT, AOT-E, Numeracy, and mindware with both persuasion and dissuasion indexes in each group and in the entire sample. The relationship between these measures should be stronger, although not significantly, for participants in the group with complex arguments.

2 Getting Ready

First, we need to design the experiment. For this experiment, we use online platforms for data collection. There are several options such as Gorilla, JSpsych, Qualtrics, psychoJS (pavlovia), etc. Since we do not need any reaction time data, we simply use Qualtrics. For an overview of different lab-based and online platforms, see here.

Next, we need to decide on the number of participants (sample size). For this study, we do not sue power analysis since we cannot access more than 120 participants. However, it is highly suggested calculate sample size using power estimation. You can find some nice tutorials on how to do that here, here, and here.

After we created the experiment and decided on the sample size, the next step is to preresigter the study. However, it would be better to do a pilot with 4 or 5 participants, clean all the data, do the desired analysis, and then pre-register the analysis and those codes. You can find the preregistration form for the current study here.

Finally, we need to restructure our project in a tidy folder with different sub-folders. Having a clean and tidy folder structure can save us! There are different formats of folder structure (for example, see here and here), but for now, we use the following structure:

3 Introduction to R

# load libraries
library(tidyverse)
library(here)
library(janitor)
library(broom)
library(afex)
library(emmeans)
library(knitr)
library(kableExtra)
library(ggsci)
library(patchwork)
library(skimr)
# install.packages("devtools")
# devtools::install_github("easystats/correlation")
library("correlation")
options(scipen=999) # turn off scientific notations
options(contrasts = c('contr.sum','contr.poly')) # set the contrast sum globally 
options(knitr.kable.NA = '')

R can be used as a calculator. For mathematical purposes, be careful of the order in which R executes the commands.

10 + 10
## [1] 20
4 ^ 2
## [1] 16
(250 / 500) * 100
## [1] 50

R is a bit flexible with spacing (but no spacing in the name of variables and words)

10+10
## [1] 20
10                 +           10
## [1] 20

R can sometimes tell that you’re not finished yet

10 +

How to create a variable? Variable assignment using <- and =. Note that R is case sensitive for everything

pay <- 250

month = 12

pay * month
## [1] 3000
salary <- pay * month

Few points in naming variables and vectors: use short, informative words, keep same method (e.g., not using capital words, use only _ or . ).

3.1 Function

Function is a set of statements combined together to perform a specific task. When we use a block of code repeatedly, we can convert it to a function. To write a function, first, you need to define it:

my_multiplier <- function(a,b){
  result = a * b
  return (result)
}

This code do nothing. To get a result, you need to call it:

my_multiplier (2,4)
## [1] 8

Fortunately, you do not need to write everything from scratch. R has lots of built-in functions that you can use:

round(54.6787)
## [1] 55
round(54.5787, digits = 2)
## [1] 54.58

Use ? before the function name to get some help. For example, ?round. You will see many functions in the rest of the workshop.

3.2 Basic Data Types in R:

function class() is used to show what is the type of a variable.

  1. Logical: TRUE, FALSE can be abbreviated as T, F. They has to be capital, ‘true’ is not a logical data:
class(TRUE)
## [1] "logical"
class(F)
## [1] "logical"
  1. Numeric: all numbers e.g. 5, 10.5, 11,37; a special type of numeric is “integer” which is numbers without decimal. Integers are always numeric, but numeric is not always integer:
class(2)
## [1] "numeric"
class(13.46)
## [1] "numeric"
  1. Character: text for example, “I love R” or “4” or “4.5”:
class("ha ha ha ha")
## [1] "character"
class("56.6")
## [1] "character"
class("TRUE")
## [1] "character"

Can we change the type of data in a variable? Yes, you need to use the function as.---()

as.numeric(TRUE)
## [1] 1
as.character(4)
## [1] "4"
as.numeric("4.5")
## [1] 4.5
as.numeric("Hello")
## Warning: NAs introduced by coercion
## [1] NA

3.3 Data Structures in R

Vector: when there are more than one number or letter stored. Use the combine function c() for that.

sale <- c(1, 2, 3,4, 5, 6, 7, 8, 9, 10) # also sale <- c(1:10)

sale <- c(1:10)

sale * sale
##  [1]   1   4   9  16  25  36  49  64  81 100

Subsetting a vector:

days <- c("Saturday", "Sunday", "Monday", "Tuesday", "Wednesday", "Thursday", "Friday")

days[2]
## [1] "Sunday"
days[-2]
## [1] "Saturday"  "Monday"    "Tuesday"   "Wednesday" "Thursday"  "Friday"
days[c(2, 3, 4)]
## [1] "Sunday"  "Monday"  "Tuesday"

3.3.1 Exercise:

Create a vector named my_vector with numbers from 0 to 1000 in it:

my_vector <- (0:1000)

mean(my_vector)
## [1] 500
median(my_vector)
## [1] 500
min(my_vector)
## [1] 0
range(my_vector)
## [1]    0 1000
class(my_vector)
## [1] "integer"
sum(my_vector)
## [1] 500500
sd(my_vector)
## [1] 289.1081

List: allows you to gather a variety of objects under one name (that is, the name of the list) in an ordered way. These objects can be matrices, vectors, data frames, even other list.

my_list = list(sale, 1, 3, 4:7, "HELLO", "hello", FALSE)
my_list
## [[1]]
##  [1]  1  2  3  4  5  6  7  8  9 10
## 
## [[2]]
## [1] 1
## 
## [[3]]
## [1] 3
## 
## [[4]]
## [1] 4 5 6 7
## 
## [[5]]
## [1] "HELLO"
## 
## [[6]]
## [1] "hello"
## 
## [[7]]
## [1] FALSE

Factor: Factors store the vector along with the distinct values of the elements in the vector as labels. The labels are always character irrespective of whether it is numeric or character. For example, variable gender with “male” and “female” entries:

gender <- c("male", "male", "male", " female", "female", "female")
gender <- factor(gender)

R now treats gender as a nominal (categorical) variable: 1=female, 2=male internally (alphabetically).

summary(gender)
##  female  female    male 
##       1       2       3

Question: why when we ran the above function i.e. summary(), it showed three and not two levels of the data? Hint: run ‘gender’.

gender
## [1] male    male    male     female female  female 
## Levels:  female female male

So, be careful of spaces!

3.3.2 Exercise:

Create a gender factor with 30 male and 40 females (Hint: use the rep() function):

gender <- c(rep("male",30), rep("female", 40))
gender <- factor(gender)
gender
##  [1] male   male   male   male   male   male   male   male   male   male  
## [11] male   male   male   male   male   male   male   male   male   male  
## [21] male   male   male   male   male   male   male   male   male   male  
## [31] female female female female female female female female female female
## [41] female female female female female female female female female female
## [51] female female female female female female female female female female
## [61] female female female female female female female female female female
## Levels: female male

There are two types of categorical variables: nominal and ordinal. How to create ordered factors (when the variable is nominal and values can be ordered)? We should add two additional arguments to the factor() function: ordered = TRUE, and levels = c("level1", "level2"). For example, we have a vector that shows participants’ education level.

edu<-c(3,2,3,4,1,2,2,3,4)

education<-factor(edu, ordered = TRUE)
levels(education) <- c("Primary school","high school","College","Uni graduated")
education
## [1] College        high school    College        Uni graduated 
## [5] Primary school high school    high school    College       
## [9] Uni graduated 
## Levels: Primary school < high school < College < Uni graduated

3.3.3 Exercise:

We have a factor with patient and control values. Here, the first level is control and the second level is patient. Change the order of levels, so patient would be the first level:

health_status <- factor(c(rep('patient',5),rep('control',5)))
health_status
##  [1] patient patient patient patient patient control control control
##  [9] control control
## Levels: control patient
health_status_reordered <- factor(health_status, levels = c('patient','control'))
health_status_reordered
##  [1] patient patient patient patient patient control control control
##  [9] control control
## Levels: patient control

Finally, can you relabel both levels to uppercase characters? (Hint: check ?factor)

health_status_relabeled <- factor(health_status, levels = c('patient','control'), labels = c('Patient','Control'))
health_status_relabeled
##  [1] Patient Patient Patient Patient Patient Control Control Control
##  [9] Control Control
## Levels: Patient Control

Matrices: All columns in a matrix must have the same mode(numeric, character, etc.) and the same length. It can be created using a vector input to the matrix function.

my_matrix = matrix(c(1,2,3,4,5,6,7,8,9), nrow = 3, ncol = 3)

my_matrix
##      [,1] [,2] [,3]
## [1,]    1    4    7
## [2,]    2    5    8
## [3,]    3    6    9

Data frames: (two-dimensional objects) can hold numeric, character or logical values. Within a column all elements have the same data type, but different columns can be of different data type. Let’s create a dataframe:

id <- 1:200
group <- c(rep("Psychotherapy", 100), rep("Medication", 100))
response <- c(rnorm(100, mean = 30, sd = 5),
             rnorm(100, mean = 25, sd = 5))

my_dataframe <-data.frame(Patient = id,
                          Treatment = group,
                          Response = response)

We also could have done the below

my_dataframe <-data.frame(Patient = c(1:200),
                          Treatment = c(rep("Psychotherapy", 100), rep("Medication", 100)),
                          Response = c(rnorm(100, mean = 30, sd = 5),
                                       rnorm(100, mean = 25, sd = 5)))

In large data sets, the function head() enables you to show the first observations of a data frames. Similarly, the function tail() prints out the last observations in your data set.

head(my_dataframe) 
tail(my_dataframe)
Patient Treatment Response
1 Psychotherapy 25.72770
2 Psychotherapy 31.21325
3 Psychotherapy 29.99580
4 Psychotherapy 27.09983
5 Psychotherapy 26.81725
6 Psychotherapy 26.74217
Patient Treatment Response
195 195 Medication 20.40254
196 196 Medication 24.84327
197 197 Medication 22.79912
198 198 Medication 33.18292
199 199 Medication 24.58559
200 200 Medication 25.31449

Similar to vectors and matrices, brackets [] are used to selects data from rows and columns in data.frames:

my_dataframe[35, 3]
## [1] 28.7423

3.3.4 Exercise

How can we get all columns, but only for the first 10 participants?

my_dataframe[1:10, ]
Patient Treatment Response
1 Psychotherapy 25.72770
2 Psychotherapy 31.21325
3 Psychotherapy 29.99580
4 Psychotherapy 27.09983
5 Psychotherapy 26.81725
6 Psychotherapy 26.74217
7 Psychotherapy 34.86773
8 Psychotherapy 36.61779
9 Psychotherapy 34.68180
10 Psychotherapy 36.49156

How to get only the Response column for all participants?

my_dataframe[ , 3]
##   [1] 25.72770 31.21325 29.99580 27.09983 26.81725 26.74217 34.86773
##   [8] 36.61779 34.68180 36.49156 30.89007 35.41123 38.85842 30.57155
##  [15] 30.42493 26.47939 27.50490 31.56233 37.30031 33.42363 29.32580
##  [22] 32.88187 34.51232 23.88507 26.51493 25.42962 28.44067 29.23889
##  [29] 25.24702 28.57276 30.52609 35.64100 25.68543 31.01235 28.74230
##  [36] 25.07624 32.76672 38.24879 16.92928 24.42460 28.02411 34.69037
##  [43] 30.40711 29.48666 37.96717 24.29767 32.64481 25.84578 29.28556
##  [50] 44.11920 25.07969 21.43629 27.52244 37.24509 38.36706 35.46408
##  [57] 23.45111 27.39520 23.59920 29.08243 32.88755 29.34568 36.76027
##  [64] 30.98235 30.61806 35.48034 27.39239 32.46603 28.87644 33.11735
##  [71] 28.45200 30.72661 27.58458 30.80198 26.08880 31.74715 34.42052
##  [78] 36.14018 29.15979 15.75634 33.51822 37.88828 26.47385 33.26013
##  [85] 34.78432 19.44502 31.93158 31.48073 31.48999 33.27916 28.62990
##  [92] 29.38271 28.57686 30.40548 34.42423 29.16725 32.40763 37.41949
##  [99] 28.13617 19.76279 25.79459 34.52733 26.44130 18.30816 27.64192
## [106] 17.47271 22.26139 24.29191 22.66851 15.63327 28.92695 20.81435
## [113] 21.88169 20.83022 33.60910 24.55989 17.62970 29.40357 30.38139
## [120] 24.33577 29.02207 16.05454 18.13544 34.35895 26.43107 20.91289
## [127] 17.87389 26.99069 25.12145 19.37052 20.88742 22.78649 13.54631
## [134] 33.21585 17.62122 25.56304 20.15252 27.47051 28.10522 24.03826
## [141] 29.80794 25.99564 21.25735 17.59239 27.46318 29.03501 30.01122
## [148] 33.74218 25.65373 26.46268 27.91304 27.24095 28.31243 22.22584
## [155] 31.01431 20.95776 28.17836 23.07360 20.12716 34.05968 22.96145
## [162] 15.87866 23.98782 19.77766 28.34237 30.83325 22.29114 24.69355
## [169] 12.81555 25.69261 23.22378 30.95828 24.23875 20.91039 32.04584
## [176] 25.42380 26.17715 25.99621 17.85196 23.01855 24.33618 25.42155
## [183] 16.25279 14.64445 22.04017 23.45330 22.37534 22.12577 24.22884
## [190] 27.32652 25.11790 18.05474 19.64780 25.75833 20.40254 24.84327
## [197] 22.79912 33.18292 24.58559 25.31449

Another easier way for selecting particular items is using their names that is more helpful than number of the rows in large data sets:

my_dataframe[ , "Response"]
# OR:
my_dataframe$Response

4 Data Cleaning

Now, suppose we tested 141 students. First, let’s read and check the uncleaned data:

# read the raw data
raw_data <- read_csv(here("raw_data","raw_argumentative_exp1.csv"))
head(raw_data)
end_date status ip_address progress duration_in_seconds subject recorded_date response_id location_latitude location_longitude distribution_channel user_language consent_form age gender stage1_simple stage2_simple stage3_simple stage4_simple stage5_simple stage6_simple stage7_simple stage1_complex stage2_complex stage3_complex stage4_complex stage5_complex stage6_complex stage7_complex attention1 attention2 attention3 crt1 crt2 crt3 aote1 aote2 aote3 aote4 aote5 aote6 aote7 aote8 group attention_correct numeracy_total mindware_total
24/9/20 22:02 IP Address 202.7.193.64 100 1517 subj1 24/9/20 22:02 R_1f298znjmVzcOjp -33.85910 151.2002 anonymous EN I consent 18 Female 36 70 68 54 51 43 41 1 1 1 8 50 20 5 5 5 5 5 6 5 3 Complex 3 9 9
25/9/20 3:23 IP Address 220.245.220.94 100 1131 subj2 25/9/20 3:23 R_tL0A9P33Gi18I0N -34.03680 150.6672 anonymous EN I consent 18 Male 50 55 55 90 75 50 35 1 1 1 8 10 39 6 6 6 5 6 6 5 6 Simple 3 9 10
27/9/20 22:59 IP Address 121.210.0.211 100 709 subj3 27/9/20 22:59 R_1LNyJhCKxTAAMOW -33.85910 151.2002 anonymous EN I consent 19 Female 50 50 77 60 25 20 13 0 1 1 8 50 20 6 5 4 5 5 6 5 6 Simple 2 10 8
27/9/20 23:18 IP Address 58.179.100.109 100 949 subj4 27/9/20 23:18 R_3enxzUsEYgs5r1a -12.63921 141.8741 anonymous EN I consent 27 Female 70 80 90 95 70 80 90 1 1 1 8 50 20 6 6 6 1 6 6 6 1 Complex 3 8 7
28/9/20 0:45 IP Address 120.154.53.68 100 1097 subj5 28/9/20 0:45 R_2Qzl2096a4KNE29 -33.85910 151.2002 anonymous EN I consent 19 Male 71 73 85 95 95 32 32 1 1 1 4 10 39 6 6 5 5 6 6 6 6 Simple 3 11 11
28/9/20 2:20 IP Address 1.129.107.6 100 880 subj6 28/9/20 2:20 R_esb71WOTQySjusF -33.85910 151.2002 anonymous EN I consent 20 Female 89 100 44 55 100 50 55 1 1 1 8 50 20 6 6 6 5 6 6 6 6 Complex 3 10 10

Now, let’s do some cleanining using dplyr, tidyr and other tidyverse libraries. Finally, we will check the data:

cleaned_data <- raw_data %>% 
  filter(progress == 100) %>% # filter out unfinished participants
  select(-end_date, -status,-ip_address, -duration_in_seconds, -recorded_date:-user_language) %>% #remove some useless columns
  mutate(aote_total= aote1+aote2+aote3+aote4+aote5+aote6+aote7+aote8, # create a total score for our questionnaire
         attentive= case_when(attention_correct >= 2~ 'Yes',T~ 'No')) %>%
  mutate(crt1= case_when(crt1=='4'~ 1,T~0),
         crt2= case_when(crt2=='10'~ 1,T~0),
         crt3= case_when(crt3=='39'~ 1,T~0),
         crt_total= crt1 + crt2 + crt3) %>%
  select(-attention1:-aote8) %>%
  pivot_longer(cols = c(stage1_simple:stage7_simple,stage1_complex:stage7_complex),names_to = 'stage',values_to = 'truth_estimate') %>% # make our dataframe long
  #pivot_wider(names_from = stage, values_from= truth_estimate) # this code change our dataframe back to wide
  filter(!is.na(truth_estimate)) %>% #remove rows with truth_estimate == NA
  mutate(stage= gsub("_.*", "", stage)) %>%
  rename(consent= consent_form) %>% # rename a column
  #mutate_if(is.character, factor) %>%
  mutate(subject= factor(subject), # convert all characters to factor
         group = factor(group),
         attentive = factor(attentive),
         stage = factor(stage))
progress subject consent age gender group attention_correct numeracy_total mindware_total aote_total attentive crt_total stage truth_estimate
100 subj1 I consent 18 Female Complex 3 9 9 39 Yes 0 stage1 36
100 subj1 I consent 18 Female Complex 3 9 9 39 Yes 0 stage2 70
100 subj1 I consent 18 Female Complex 3 9 9 39 Yes 0 stage3 68
100 subj1 I consent 18 Female Complex 3 9 9 39 Yes 0 stage4 54
100 subj1 I consent 18 Female Complex 3 9 9 39 Yes 0 stage5 51
100 subj1 I consent 18 Female Complex 3 9 9 39 Yes 0 stage6 43

Ok, now the data is clean and tidy which means:

  1. Each variable forms a column.
  2. Each observation forms a row.
  3. Each type of observational unit forms a table (Wickham, 2014).

Check the dataframe and all the data types:

str(cleaned_data)
## tibble [917 × 14] (S3: tbl_df/tbl/data.frame)
##  $ progress         : num [1:917] 100 100 100 100 100 100 100 100 100 100 ...
##  $ subject          : Factor w/ 131 levels "subj1","subj10",..: 1 1 1 1 1 1 1 45 45 45 ...
##  $ consent          : chr [1:917] "I consent" "I consent" "I consent" "I consent" ...
##  $ age              : num [1:917] 18 18 18 18 18 18 18 18 18 18 ...
##  $ gender           : chr [1:917] "Female" "Female" "Female" "Female" ...
##  $ group            : Factor w/ 2 levels "Complex","Simple": 1 1 1 1 1 1 1 2 2 2 ...
##  $ attention_correct: num [1:917] 3 3 3 3 3 3 3 3 3 3 ...
##  $ numeracy_total   : num [1:917] 9 9 9 9 9 9 9 9 9 9 ...
##  $ mindware_total   : num [1:917] 9 9 9 9 9 9 9 10 10 10 ...
##  $ aote_total       : num [1:917] 39 39 39 39 39 39 39 46 46 46 ...
##  $ attentive        : Factor w/ 2 levels "No","Yes": 2 2 2 2 2 2 2 2 2 2 ...
##  $ crt_total        : num [1:917] 0 0 0 0 0 0 0 2 2 2 ...
##  $ stage            : Factor w/ 7 levels "stage1","stage2",..: 1 2 3 4 5 6 7 1 2 3 ...
##  $ truth_estimate   : num [1:917] 36 70 68 54 51 43 41 50 55 55 ...

Finally, we save our data to the cleaned_data folder.

write_csv(cleaned_data, here("cleaned_data","argumentative_exp1.csv"))

5 Descriptive Statistics

Note: All the data that we use here is manipulated (fabricated) for teaching purpuses. In our study, we failed to find such beautiful and interesting results.

Now, let’s do some descriptive statistics. First, we can open a new script called analysis_exp1.r and read the cleaned data again.

data_exp1 <- read_csv(here("cleaned_data","argumentative_exp1.csv"))

How many participants in total?

data_exp1 %>% summarise(n= n_distinct(subject))
n
131

how many participants in each group?

data_exp1 %>% 
  group_by(subject) %>% 
  filter(row_number()==1) %>% 
  ungroup () %>% 
  group_by(group) %>% 
  count() 
group n
Complex 65
Simple 66

Find the mean and sd for numeric variables using base R summary function:

data_exp1 %>% 
  group_by(subject) %>% 
  filter(row_number()==1) %>% 
  ungroup () %>%
  summary()
##     progress     subject            consent               age       
##  Min.   :100   Length:131         Length:131         Min.   :16.00  
##  1st Qu.:100   Class :character   Class :character   1st Qu.:18.00  
##  Median :100   Mode  :character   Mode  :character   Median :19.00  
##  Mean   :100                                         Mean   :21.15  
##  3rd Qu.:100                                         3rd Qu.:20.00  
##  Max.   :100                                         Max.   :63.00  
##     gender             group           attention_correct numeracy_total  
##  Length:131         Length:131         Min.   :0.000     Min.   : 0.000  
##  Class :character   Class :character   1st Qu.:2.000     1st Qu.: 8.000  
##  Mode  :character   Mode  :character   Median :3.000     Median :10.000  
##                                        Mean   :2.573     Mean   : 8.779  
##                                        3rd Qu.:3.000     3rd Qu.:10.000  
##                                        Max.   :3.000     Max.   :11.000  
##  mindware_total    aote_total    attentive           crt_total     
##  Min.   : 4.00   Min.   :19.0   Length:131         Min.   :0.0000  
##  1st Qu.: 8.00   1st Qu.:33.5   Class :character   1st Qu.:0.0000  
##  Median : 8.00   Median :39.0   Mode  :character   Median :0.0000  
##  Mean   : 8.45   Mean   :38.2                      Mean   :0.8092  
##  3rd Qu.: 9.00   3rd Qu.:43.0                      3rd Qu.:1.0000  
##  Max.   :12.00   Max.   :48.0                      Max.   :3.0000  
##     stage           truth_estimate  
##  Length:131         Min.   :  0.00  
##  Class :character   1st Qu.: 43.50  
##  Mode  :character   Median : 61.00  
##                     Mean   : 57.09  
##                     3rd Qu.: 75.50  
##                     Max.   :100.00

Alternatively, we can use base Rsummaryfunctionskimr` library:

data_exp1 %>% 
  group_by(subject) %>% 
  filter(row_number()==1) %>% 
  ungroup () %>% 
  dplyr::select (age, numeracy_total, mindware_total, aote_total, crt_total) %>% 
  skimr::skim()
skim_type skim_variable n_missing complete_rate numeric.mean numeric.sd numeric.p0 numeric.p25 numeric.p50 numeric.p75 numeric.p100 numeric.hist
numeric age 0 1 21.1526718 6.515630 16 18.0 19 20 63 ▇▁▁▁▁
numeric numeracy_total 0 1 8.7786260 2.274576 0 8.0 10 10 11 ▁▁▁▂▇
numeric mindware_total 0 1 8.4503817 1.683466 4 8.0 8 9 12 ▁▅▇▆▃
numeric aote_total 0 1 38.1984733 6.153698 19 33.5 39 43 48 ▁▂▇▇▆
numeric crt_total 0 1 0.8091603 1.038598 0 0.0 0 1 3 ▇▃▁▂▂

5.0.1 Exercise

For this exercise, we use a dataset of one of my own studies. In this study, we asked participants to guess the physical brightness of reasoning arguments and then we gave a cognitive ability test. (See the original study here). Open ghasemi_brightness_exp4.csv file and answer to the following questions:

  1. How many participants did we test in total?
  2. Find out how many male and female we tested.
  3. Calculate mean and sd for age and cognitive ability (ah4).
ghasemi_data <- read_csv(here("cleaned_data","ghasemi_brightness_exp4.csv"))

ghasemi_data %>% summarise(n = n_distinct(participant)) # number of participants:200
n
200
ghasemi_data %>% group_by (participant) %>% filter (row_number()==1) %>% group_by (gender) %>% summarise(n= n()) %>% ungroup() # 183 female, 17 male
gender n
Female 183
Male 17
ghasemi_data %>% dplyr::select (age, ah4) %>% skimr::skim() # mean and sd for age and cognitive ability
Data summary
Name Piped data
Number of rows 38400
Number of columns 2
_______________________
Column type frequency:
numeric 2
________________________
Group variables

Variable type: numeric

skim_variable n_missing complete_rate mean sd p0 p25 p50 p75 p100 hist
age 0 1 22.20 6.78 17 19 20 22 52 ▇▁▁▁▁
ah4 0 1 39.55 9.46 11 34 40 46 61 ▁▃▇▆▂

6 Data Visualization

First, we need to create a dataset with aggregated truth estimate scores over group and stage. We will use this dataset for line and bar graphs.

aggregated_data_exp1 <- data_exp1 %>%
  group_by(stage, group) %>%
  mutate(truth_estimate = mean(truth_estimate)) %>%
  ungroup()

barplot_exp1 <- aggregated_data_exp1 %>%
  ggplot(aes(x=stage, y= truth_estimate, fill=group)) +
  geom_bar(stat = "identity", position= "dodge")+
  # stat_summary(fun= mean, geom = "bar", position = "dodge")+ # can be used instead of geom_bar() for long dataframes
  labs (x= '', y= "Truth Likelihhod Estimate") + 
  theme_bw() + 
  scale_fill_jama() 

barplot_exp1

barplot_facet_exp1 <- aggregated_data_exp1 %>%
  ggplot(aes(x=group, y= truth_estimate, fill=stage)) +
  geom_bar(stat = "identity", position= "dodge")+
  labs (x= '', y= "Truth Likelihhod Estimate") + 
  theme_bw() + 
  theme(legend.position = "none",
        axis.text=element_text(size=11),
        axis.title = element_text(size = 12)) +
  facet_wrap(~stage)+
  scale_fill_jco() 

barplot_facet_exp1

lineplot_exp1 <- aggregated_data_exp1 %>%
  ggplot(aes(x=factor(stage), y= truth_estimate, group= group, color= group)) +
  geom_line(aes(linetype= group)) +
  geom_point(size= 5)+
  labs (x= '', y= "Truth Likelihhod Estimate") + 
  theme_classic() +
  theme(legend.position = "bottom",
        axis.text=element_text(size=11),
        axis.title = element_text(size = 12)) +
  scale_color_nejm() 

lineplot_exp1

violinplot_exp1 <- data_exp1 %>%
  ggplot(aes(x=factor(stage), y= truth_estimate, fill= group)) +
  geom_violin()+
  labs (x= '', y= "Truth Likelihhod Estimate") + 
  theme_bw() + 
  theme(legend.position = "bottom",
        axis.text=element_text(size=11),
        axis.title = element_text(size = 12)) +
  scale_fill_d3() 

violinplot_exp1

boxplot_exp1 <- data_exp1 %>%
  ggplot(aes(x=factor(stage), y= truth_estimate, fill= group)) +
  geom_boxplot()+
  #geom_point(position = position_dodge(width=0.75), alpha= .5)+
  labs (x= '', y= "Truth Likelihhod Estimate") + 
  theme_bw() + 
  theme(legend.position = "bottom",
        axis.text=element_text(size=11),
        axis.title = element_text(size = 12)) +
  scale_fill_simpsons() 

boxplot_exp1

boxplot_facet_exp1 <- data_exp1 %>%
  ggplot(aes(x=factor(stage), y= truth_estimate, fill= group)) +
  geom_boxplot()+
  labs (x= '', y= "Truth Likelihhod Estimate") + 
  theme_bw() + 
  theme(legend.position = "bottom",
        axis.text=element_text(size=11),
        axis.title = element_text(size = 12),
        axis.text.x = element_text(angle = 90, vjust = 0.5, hjust=1)) +
  facet_wrap(~group)+
  scale_color_simpsons() 

boxplot_facet_exp1

How to combine multiple plots? We can use the patchwork package. A nice tutorial on using this package can be found here

combined_plot_exp1 <- (barplot_facet_exp1+lineplot_exp1) / (violinplot_exp1+boxplot_exp1)
combined_plot_exp1

How to save a plot?

ggsave(combined_plot_exp1, filename = here("outputs","combined_plot_exp1.png"), dpi=300)

7 Data Analysis

7.1 t-test

Is there a difference between groups at the first stage? Ideally, we want participants’ ratings at the first stage be similar for both groups because we have not done any manipulations. Previous graphs showed us that ratings of simple and complex group at this stage are pretty close. Let’s test that using an independent t-test (because we have 2 independent groups):

# Is there a difference between groups at the first stage?
data_exp1 %>% 
  group_by(group) %>% 
  filter(stage=='stage1') %>% 
  ungroup () %>%
  t.test(truth_estimate~group, data = ., paired=FALSE)
## 
##  Welch Two Sample t-test
## 
## data:  truth_estimate by group
## t = -0.75145, df = 104.95, p-value = 0.4541
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -11.883716   5.351781
## sample estimates:
## mean in group Complex  mean in group Simple 
##              55.44615              58.71212

Now, we wonder if opposing arguments were effective at all, regardless of participants’ group. So, we would like to test if ratings at the final stage are lower than ratings at the stage 4? Since a pair of score at stage 4 and stage 7 is coming from a same person, we use paired t-test.

# Is there a difference between ratings of stage4 and stage7?
data_exp1 %>% 
  filter(stage=='stage4' | stage=='stage7') %>% 
  ungroup () %>%
  t.test(truth_estimate~stage, data = ., paired=TRUE)
## 
##  Paired t-test
## 
## data:  truth_estimate by stage
## t = 12.788, df = 130, p-value < 0.00000000000000022
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  32.64368 44.59296
## sample estimates:
## mean of the differences 
##                38.61832

7.1.1 Exercise

John et al. (2019) investigated the consequences of backing down (changing one’s mind in lights of evidence)and how other people view someone who change their mind. In their second experiments, they presented participants either with a person who changes their mind or a person who refuses to back down. Then, they asked participants to rate how intelligent and confident the person is (See the original study here). They reported that:

“Relative to the entrepreneur who did not back down, participants judged the entrepreneur who backed down as more intelligent (M_backed_down=5.13 out of 7, SD=1.09; M_did_not_back_down=3.97, SD=1.54; t(271.12)=−7.59, p < .001) but less confident (M_backed_down=4.50 out of 7, SD=1.36; M_did_not_back_down=5.65, SD=1.10; t(291.01)=8.08, p < .001).”.

Open the john_backdown_exp2.csv file and try to reproduce their results. Run two separate independent t-test, one with intelligent as the dependent variable and one with confident as the dependent variable. For both t-test, use back_down as the between-subject independent variable.

john_data <- read_csv(here("cleaned_data","john_backdown_exp2.csv"))


t.test(intelligent~back_down, data = john_data, paired=FALSE)
## 
##  Welch Two Sample t-test
## 
## data:  intelligent by back_down
## t = 7.5853, df = 271.12, p-value = 0.0000000000005319
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  0.8577107 1.4590076
## sample estimates:
##       mean in group backed_down mean in group did_not_back_down 
##                        5.129412                        3.971053
t.test(confident~back_down, data = john_data, paired=FALSE)
## 
##  Welch Two Sample t-test
## 
## data:  confident by back_down
## t = -8.0763, df = 291.01, p-value = 0.00000000000001787
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -1.4257768 -0.8670294
## sample estimates:
##       mean in group backed_down mean in group did_not_back_down 
##                        4.503268                        5.649671

7.2 Analysis of Variance (ANOVA)

Now, let’s answer our main question: Do participants in the simple group show higher ratings for supportive arguments (stage 2 to 4) and lower ratings for opposing arguments (stage 5 to 7), compared to participants in the complex group? If this is the case. we expect an interaction in the traditional Analysis of Variance (AONVA) test.

aov_m1 <- aov_car (truth_estimate ~ group*stage +
                     Error(subject/stage), data = data_exp1)
Effect df MSE F ges p.value
group 1, 129 949.04 0.01 <.0001 .94
stage 4.45, 574.05 515.69 59.48 *** .25 <.0001
group:stage 4.45, 574.05 515.69 13.34 *** .07 <.0001

As you can see, we found a significant main effect of stage and a significant group by stage interaction. We can use the emmeans package to do post-hoc tests.

# main effect of stage
emmeans(aov_m1, 'stage')
##  stage  emmean   SE  df lower.CL upper.CL
##  stage1   57.1 1.88 763     53.4     60.8
##  stage2   66.8 1.88 763     63.1     70.5
##  stage3   74.6 1.88 763     70.9     78.3
##  stage4   79.6 1.88 763     75.9     83.3
##  stage5   62.4 1.88 763     58.7     66.1
##  stage6   52.5 1.88 763     48.9     56.2
##  stage7   41.1 1.88 763     37.4     44.8
## 
## Results are averaged over the levels of: group 
## Warning: EMMs are biased unless design is perfectly balanced 
## Confidence level used: 0.95
pairs(emmeans(aov_m1, 'stage'), adjust= 'holm')
##  contrast        estimate   SE  df t.ratio p.value
##  stage1 - stage2    -9.74 2.42 774 -4.031  0.0004 
##  stage1 - stage3   -17.53 2.42 774 -7.256  <.0001 
##  stage1 - stage4   -22.50 2.42 774 -9.311  <.0001 
##  stage1 - stage5    -5.29 2.42 774 -2.187  0.1160 
##  stage1 - stage6     4.53 2.42 774  1.876  0.1220 
##  stage1 - stage7    15.98 2.42 774  6.613  <.0001 
##  stage2 - stage3    -7.79 2.42 774 -3.225  0.0066 
##  stage2 - stage4   -12.76 2.42 774 -5.280  <.0001 
##  stage2 - stage5     4.46 2.42 774  1.844  0.1220 
##  stage2 - stage6    14.28 2.42 774  5.908  <.0001 
##  stage2 - stage7    25.72 2.42 774 10.644  <.0001 
##  stage3 - stage4    -4.97 2.42 774 -2.055  0.1206 
##  stage3 - stage5    12.25 2.42 774  5.069  <.0001 
##  stage3 - stage6    22.07 2.42 774  9.132  <.0001 
##  stage3 - stage7    33.51 2.42 774 13.869  <.0001 
##  stage4 - stage5    17.22 2.42 774  7.124  <.0001 
##  stage4 - stage6    27.04 2.42 774 11.188  <.0001 
##  stage4 - stage7    38.48 2.42 774 15.924  <.0001 
##  stage5 - stage6     9.82 2.42 774  4.064  0.0004 
##  stage5 - stage7    21.27 2.42 774  8.800  <.0001 
##  stage6 - stage7    11.45 2.42 774  4.736  <.0001 
## 
## Results are averaged over the levels of: group 
## P value adjustment: holm method for 21 tests
# group by stage interaction
emmeans(aov_m1, "group", by= "stage")
## stage = stage1:
##  group   emmean   SE  df lower.CL upper.CL
##  Complex   55.4 2.67 766     50.2     60.7
##  Simple    58.7 2.65 761     53.5     63.9
## 
## stage = stage2:
##  group   emmean   SE  df lower.CL upper.CL
##  Complex   63.3 2.67 766     58.1     68.6
##  Simple    70.3 2.65 761     65.1     75.5
## 
## stage = stage3:
##  group   emmean   SE  df lower.CL upper.CL
##  Complex   70.0 2.67 766     64.7     75.2
##  Simple    79.3 2.65 761     74.1     84.5
## 
## stage = stage4:
##  group   emmean   SE  df lower.CL upper.CL
##  Complex   71.6 2.67 766     66.3     76.8
##  Simple    87.6 2.65 761     82.4     92.8
## 
## stage = stage5:
##  group   emmean   SE  df lower.CL upper.CL
##  Complex   64.2 2.67 766     58.9     69.4
##  Simple    60.5 2.65 761     55.3     65.8
## 
## stage = stage6:
##  group   emmean   SE  df lower.CL upper.CL
##  Complex   57.9 2.67 766     52.7     63.2
##  Simple    47.2 2.65 761     41.9     52.4
## 
## stage = stage7:
##  group   emmean   SE  df lower.CL upper.CL
##  Complex   51.1 2.67 766     45.9     56.4
##  Simple    31.1 2.65 761     25.9     36.3
## 
## Warning: EMMs are biased unless design is perfectly balanced 
## Confidence level used: 0.95
update(pairs(emmeans(aov_m1, "group", by= "stage")), by = NULL, adjust = "holm") 
##  contrast         stage  estimate   SE  df t.ratio p.value
##  Complex - Simple stage1    -3.27 3.76 763 -0.868  0.6673 
##  Complex - Simple stage2    -6.96 3.76 763 -1.851  0.1935 
##  Complex - Simple stage3    -9.29 3.76 763 -2.469  0.0550 
##  Complex - Simple stage4   -16.02 3.76 763 -4.259  0.0001 
##  Complex - Simple stage5     3.64 3.76 763  0.967  0.6673 
##  Complex - Simple stage6    10.79 3.76 763  2.868  0.0213 
##  Complex - Simple stage7    20.08 3.76 763  5.337  <.0001 
## 
## P value adjustment: holm method for 7 tests

You can use the afex_plot function from afex to create beautiful plots. Those plots interacts nicely with ggplot:

afex_plot(aov_m1, x = "stage", trace = "group", error='between',
          line_arg = list(size=1),
          point_arg = list(size=3.5),
          data_arg = list(size= 1, color= 'grey', width=.4),
          data_geom = geom_boxplot,
          mapping = c("linetype", "shape", "fill"),
          legend_title = "Group") +
  labs(y = "Truth Likelihhod Estimate", x = "") +
  theme_bw()+ # remove the grey background and grid
  theme(axis.text=element_text(size=13),
        axis.title = element_text(size = 13),
        legend.text=element_text(size=13),
        legend.title=element_text(size=13),
        legend.position='bottom',
        legend.key.size = unit(1, "cm"),
        legend.background = element_rect(colour = 'black', fill = 'white', linetype='solid'))+
  scale_color_simpsons() +
  scale_fill_simpsons()

If you are interested in this topic, check out this nice tutorial about using afex to run ANOVA, and also this interesting tutorial on the emmeans package.

7.2.1 Exercise

Rotello et al. (2018) investigated the association between the race (White vs. Black faces) and the gun-tool judgments. In their first experiments, they presented participants with 16 White male faces and 16 Black male faces, and following that 8 images of guns and 8 images of tools. They asked participants to judge if the object is a tool or a gun by pressing keyboard buttons. Then, they ran an ANOVA to see if participants’ gun responses are higher for any of the races. So, they included prime race (Black, White) and target identity (gun, tool) as independent variables and participants’ gun responses as dependent variable into their linear model (See the original study here). They found that:

“Participants made more gun responses to guns than to tools, F(1,45) = 53243, p < 0.0001, η2g = 0.998. However, the race of the prime face did not matter, F(1,45) = 0.287, p > 0.59, η2g = 0.001, nor was there an interaction of prime race with target object, F(1,45) = 0.022, p > 0.88, η2g = 0.000)”.

Open the rotello_shooter_exp1.csv file and try to reproduce their results. Run an ANOVA (type III) with resp as the dependent variable and target, prime, and their interaction as independent variables.

# load the general data file
rotello_data <- read_csv(here("cleaned_data","rotello_shooter_exp1.csv"))

# ANOVA
rotello_aov <- aov_car (resp ~ target*prime +
           Error(subject/target*prime), data = rotello_data)
Effect df MSE F ges p.value
target 1, 45 0.00 53242.99 *** >.99 <.0001
prime 1, 45 0.00 0.29 .001 .59
target:prime 1, 45 0.00 0.02 <.0001 .88

7.3 Correlation

Now, let’s answer to another question of this study: does persuasion and dissuasion is related to open-mindedness, cognitive ability, reasoning abilities, and cognitive style? To answer this question, we need to create two indexes (scores) one for persuasion and one for dissuasion. Then we can do a correlation test:

cor_data_exp1 <- data_exp1 %>% 
  pivot_wider(names_from = stage, values_from = truth_estimate) %>%
  group_by(subject) %>%
  mutate(persuasion_index= stage2+ stage3+ stage4 - stage1,
         dissuasion_index= (101-stage5) + (101-stage6) + (101-stage7) - (101-stage4)) %>%
  ungroup()%>%
  dplyr::select(persuasion_index,dissuasion_index,aote_total,numeracy_total,crt_total,mindware_total)

#---------- Base R:
cor(cor_data_exp1, method = "pearson",  use = "complete.obs")

#---------- Psych library:
cor_data_exp1 %>% 
  psych::pairs.panels(method = "pearson", hist.col = "#00AFBB", density = T, ellipses = F, stars = T)

#---------- Correlation library:
correlation::correlation(cor_data_exp1) %>% summary()

#---------- apaTables library:
cor_data_exp1 %>% 
  apaTables::apa.cor.table(filename="./outputs/CorMatrix.doc", show.conf.interval=T)
persuasion_index dissuasion_index aote_total numeracy_total crt_total mindware_total
persuasion_index 1.00 0.26 0.25 0.16 0.16 0.11
dissuasion_index 0.26 1.00 -0.03 -0.03 -0.09 0.15
aote_total 0.25 -0.03 1.00 0.40 0.26 0.11
numeracy_total 0.16 -0.03 0.40 1.00 0.44 0.15
crt_total 0.16 -0.09 0.26 0.44 1.00 0.29
mindware_total 0.11 0.15 0.11 0.15 0.29 1.00
Parameter mindware_total crt_total numeracy_total aote_total dissuasion_index
persuasion_index 0.11 0.16 0.16 0.25 0.26
dissuasion_index 0.15 -0.09 -0.03 -0.03
aote_total 0.11 0.26 0.40
numeracy_total 0.15 0.44
crt_total 0.29

7.3.1 Exercise

Pennycook et al. (2020) investigated the relationship between actively open-minded thinking style about evidence (AOT-E) and different political, scientific, and religious beliefs (see the original paper here). In their first experiment, they calculated the correlation of AOTE and scientific beliefs items (global warming, evolution, etc.) and they found the following results:

Open the pennycook_aote_exp1.csv file and try to reproduce their results by creating the same correlation matrix.

pennycook_data <- read_csv(here("cleaned_data","pennycook_aote_exp1.csv")) 


#---------- Base R:
cor(pennycook_data, method = "pearson",  use = "complete.obs")

#---------- Psych library:
pennycook_data %>% 
  psych::pairs.panels(method = "pearson", hist.col = "#00AFBB", density = T, ellipses = F, stars = T)

#---------- Correlation library:
correlation::correlation(pennycook_data) %>% summary()

#---------- apaTables library:
pennycook_data %>% 
  apaTables::apa.cor.table(filename="./outputs/CorMatrix.doc", show.conf.interval=T)
Parameter trust_scien gm_health tech_problems modern_medicine old_earth vaccines stem_cell big_bang evolution global_warming
aote 0.35 0.36 0.44 0.33 0.40 0.47 0.45 0.51 0.51 0.37
global_warming 0.42 0.06 0.14 0.18 0.33 0.26 0.31 0.33 0.38
evolution 0.48 0.33 0.28 0.36 0.47 0.39 0.54 0.78
big_bang 0.49 0.37 0.28 0.36 0.45 0.37 0.54
stem_cell 0.47 0.34 0.36 0.47 0.40 0.40
vaccines 0.43 0.52 0.49 0.53 0.38
old_earth 0.29 0.24 0.21 0.33
modern_medicine 0.43 0.42 0.47
tech_problems 0.33 0.39
gm_health 0.31

7.4 Linear Regression

In the previous section, we found that open-mindedness (AOT-E) is correlated with persuasion. Now, one may ask if open-mindedness can predict persuasion after controlling for reasoning and controlling abilities? To answer that, we can run a multiple regression analysis:

exp1_reg=lm(persuasion_index ~ aote_total+ numeracy_total+ crt_total+ mindware_total,
                  data=cor_data_exp1)
term estimate std.error statistic p.value
(Intercept) 78.57 33.08 2.38 0.02
aote_total 1.62 0.72 2.23 0.03
numeracy_total 0.72 2.11 0.34 0.73
crt_total 3.09 4.51 0.68 0.49
mindware_total 1.77 2.52 0.70 0.48

7.4.1 Exercise

Trémolière and Djeriouat (2020) examined the role of cognitive reflection and belief in science in climate change skepticism. In their first study, they revealed that cognitive reflection and belief in science negetively predicted climate change skepticism even after controlling for demographic and cognitive ability variables (see the original paper here).

Open the tremoliere_data_exp1.csv file and try to reproduce their results by running a multiple linear regression. Enter age, gender, education, belief in science, literacy, numeracy (Numtotal), and cognitive reflection as predictors and enter climate change skepticism (climato) as the outcome variable.

Tremoliere_data <- read_csv(here("cleaned_data","tremoliere_data_exp1.csv"))

Tremoliere_reg=lm(Climato ~ Age+ Gender+ Education+ BeliefInSciencetotal+ Literacy+ Numtotal+ CognitiveReflection,
                    data=Tremoliere_data)
term estimate std.error statistic p.value
(Intercept) 57.57 5.19 11.09 0.00
Age 0.01 0.05 0.24 0.81
Gender -5.68 1.34 -4.23 0.00
Education 0.54 0.38 1.43 0.15
BeliefInSciencetotal -0.20 0.06 -3.62 0.00
Literacy -0.49 0.51 -0.96 0.34
Numtotal -1.52 0.83 -1.82 0.07
CognitiveReflection -18.58 4.26 -4.37 0.00
r.squared adj.r.squared sigma statistic p.value df logLik AIC BIC deviance df.residual nobs
0.19 0.17 12.65 11.91 0 7 -1467.77 2953.54 2988.81 58235.89 364 372

8 Rmarkdown

To be completed…

9 References

  • Ghasemi, O., Handley, S., & Howarth, S. (2020). The Bright Homunculus in our Head: Individual Differences in Intuitive Sensitivity to Logical Validity.

  • John, L. K., Jeong, M., Gino, F., & Huang, L. (2019). The self-presentational consequences of upholding one’s stance in spite of the evidence. Organizational Behavior and Human Decision Processes, 154, 1-14.

  • Pennycook, G., Cheyne, J. A., Koehler, D. J., & Fugelsang, J. A. (2020). On the belief that beliefs should change according to evidence: Implications for conspiratorial, moral, paranormal, political, religious, and science beliefs. Judgment and Decision Making, 15(4), 476.

  • Rotello, C. M., Kelly, L. J., Heit, E., Vazire, S., & Vul, E. (2018). The Shape of ROC Curves in Shooter Tasks: Implications for Best Practices in Analysis. Collabra: Psychology, 4(1).

  • Trémolière, B., & Djeriouat, H. (2020). Don’t you see that its cold! Exploring the roles of cognitive reflection, climate science literacy, illusion of knowledge, and political orientation in climate change skepticism.

  • Wickham, H. (2014). Tidy data. Journal of Statistical Software, 59(10), 1-23.

LS0tCnRpdGxlOiAiUiBmb3IgQ29nbml0aXZlIFBzeWNob2xvZ2lzdHMiCmF1dGhvcjoKICAtIG5hbWU6ICJPbWlkIEdoYXNlbWkiCiAgICBhZmZpbGlhdGlvbjogTWFjcXVhcmllIFVuaXZlcnNpdHkKICAgIGVtYWlsOiBvbWlkcmV6YS5naGFzZW1pQGhkci5tcS5lZHUuYXUKICAtIG5hbWU6ICJNYWhkaSBNYXppZGkiCiAgICBhZmZpbGlhdGlvbjogVW5pdmVyc2l0eSBvZiBXZXN0ZXJuIEF1c3RyYWxpYQogICAgZW1haWw6IG1haGRpLm1hemlkaXNoYXJhZmFiYWRpQHJlc2VhcmNoLnV3YS5lZHUuYXUKZGF0ZTogImByIGZvcm1hdChTeXMudGltZSgpLCAnJWQgJUIsICVZJylgIgpvdXRwdXQ6IAogIGh0bWxfZG9jdW1lbnQ6CiAgICBrZWVwX21kOiB5ZXMKICAgIG51bWJlcl9zZWN0aW9uczogdHJ1ZQogICAgdGhlbWU6IGNlcnVsZWFuCiAgICBjb2RlX2Rvd25sb2FkOiB0cnVlCiAgICAjY29kZV9mb2xkaW5nOiBoaWRlCiAgICB0b2M6IHRydWUKICAgIHRvY19mbG9hdDogdHJ1ZQogICAgZGZfcHJpbnQ6ICJrYWJsZSIKLS0tCgpUaGlzIGRvY3VtZW50IGlzIHRoZSBzdW1tYXJ5IG9mIHRoZSAqKlIgZm9yIENvZ25pdGl2ZSBQc3ljaG9sb2dpc3RzKiogd29ya3Nob3AuIAoKQWxsIGNvcnJlc3BvbmRlbmNlIHJlbGF0ZWQgdG8gdGhpcyBkb2N1bWVudCBzaG91bGQgYmUgYWRkcmVzc2VkIHRvOiAKCjxjZW50ZXI+Ck9taWQgR2hhc2VtaSAoTWFjcXVhcmllIFVuaXZlcnNpdHksIFN5ZG5leSwgTlNXLCAyMTA5LCBBVVNUUkFMSUEpIAoKRW1haWw6IG9taWRyZXphLmdoYXNlbWlAaGRyLm1xLmVkdS5hdSAKPC9jZW50ZXI+CgoKCjxzdHlsZT4KCmJvZHl7IC8qIE5vcm1hbCAgKi8KICAgICAgZm9udC1zaXplOiAxOHB4OwogICAgICB0ZXh0LWFsaWduOiBqdXN0aWZ5OwogICAgICBsaW5lLWhlaWdodDogMS42OwogICAgICBmb250LWZhbWlseTogIlRpbWVzIE5ldyBSb21hbiIsIFRpbWVzLCBzZXJpZjsKfQpjb2RlLnJ7IC8qIENvZGUgYmxvY2sgKi8KICAgIGZvbnQtc2l6ZTogMTRweDsKfQpwcmUgeyAvKiBDb2RlIGJsb2NrIC0gZGV0ZXJtaW5lcyBjb2RlIHNwYWNpbmcgYmV0d2VlbiBsaW5lcyAqLwogICAgZm9udC1zaXplOiAxMnB4Owp9Cgo8L3N0eWxlPgoKCmBgYHtyIHNldHVwLCBpbmNsdWRlPUZBTFNFfQprbml0cjo6b3B0c19jaHVuayRzZXQoZWNobyA9IFRSVUUpCmtuaXRyOjpvcHRzX2NodW5rJHNldChmaWcuYWxpZ249ImNlbnRlciIpCmBgYAoKCgpgYGB7ciBsaWJyYXJpZXMsIG1lc3NhZ2U9RkFMU0UsIGVjaG89Rn0KIyBsb2FkIGxpYnJhcmllcwpsaWJyYXJ5KHRpZHl2ZXJzZSkKbGlicmFyeShoZXJlKQpsaWJyYXJ5KGphbml0b3IpCmxpYnJhcnkoYnJvb20pCmxpYnJhcnkoYWZleCkKbGlicmFyeShlbW1lYW5zKQpsaWJyYXJ5KGtuaXRyKQpsaWJyYXJ5KGthYmxlRXh0cmEpCmxpYnJhcnkoZ2dzY2kpCmxpYnJhcnkocGF0Y2h3b3JrKQpsaWJyYXJ5KHNraW1yKQojIGluc3RhbGwucGFja2FnZXMoImRldnRvb2xzIikKIyBkZXZ0b29sczo6aW5zdGFsbF9naXRodWIoImVhc3lzdGF0cy9jb3JyZWxhdGlvbiIpCmxpYnJhcnkoImNvcnJlbGF0aW9uIikKb3B0aW9ucyhzY2lwZW49OTk5KSAjIHR1cm4gb2ZmIHNjaWVudGlmaWMgbm90YXRpb25zCm9wdGlvbnMoY29udHJhc3RzID0gYygnY29udHIuc3VtJywnY29udHIucG9seScpKSAjIHNldCB0aGUgY29udHJhc3Qgc3VtIGdsb2JhbGx5IApvcHRpb25zKGtuaXRyLmthYmxlLk5BID0gJycpCmBgYAoKCiMgUmVzZWFyY2ggUXVlc3Rpb24KCgpUaGUgYWltIG9mIHRoZSBzdHVkeSBpcyB0byB0ZXN0IGlmIHNpbXBsZSBhcmd1bWVudHMgYXJlIG1vcmUgZWZmZWN0aXZlIGluIGJlbGllZiByZXZpc2lvbiB0aGFuIG1vcmUgY29tcGxleCBhcmd1bWVudHMuIFRvIHRoYXQgZW5kLCB3ZSBwcmVzZW50IHBhcnRpY2lwYW50cyB3aXRoIGFuIGltYWdpbmFyeSBzY2VuYXJpbyAodHdvIGFsaWVuIGNyZWF0dXJlcyBvbiBhIHBsYW5ldCkgYW5kIGEgdGhlb3J5IChvbmUgY3JlYXR1cmUgaXMgcHJlZGF0b3IgYW5kIHRoZSBvdGhlciBvbmUgaXMgcHJleSkgYW5kIHdlIGFzayB0aGVtIHRvIHJhdGUgdGhlIGxpa2VsaWhvb2QgdHJ1dGggb2YgdGhlIHRoZW9yeSBiYXNlZCBvbiBhIHNpbXBsZSBmYWN0IChXZSBhZGFwdGVkIHRoaXMgbWV0aG9kIGZyb20gR3JlZ2cgZXQgYWwuLDIwMTc7IHNlZSB0aGUgb3JpZ2luYWwgc3R1ZHkgW2hlcmVdKGh0dHBzOi8vam91cm5hbHMuc2FnZXB1Yi5jb20vZG9pLzEwLjEwODAvMTc0NzAyMTguMjAxNS4xMDk5MTYyKSkuIFRoZW4sIGluIGEgYmV0d2Vlbi1zdWJqZWN0IG1hbmlwdWxhdGlvbiwgcGFydGljaXBhbnRzIHdpbGwgYmUgcHJlc2VudGVkIHdpdGggZWl0aGVyIDYgc2ltcGxlIGFyZ3VtZW50cyAoTW9kdXMgUG9uZW5zIGNvbmRpdGlvbmFscykgb3IgNiBtb3JlIGNvbXBsZXggYXJndW1lbnRzIChNb2R1cyBUb2xsZW5zIGNvbmRpdGlvbmFscyksIGFuZCB0aGV5IHdpbGwgYmUgYXNrZWQgdG8gcmF0ZSB0aGUgbGlrZWxpaG9vZCB0cnV0aCBvZiB0aGUgaW5pdGlhbCB0aGVvcnkgb24gNyBzdGFnZXMuIAoKVGhlIGZpcnN0IHN0YWdlIGlzIHRoZSBiYXNlIHJhdGluZyBzdGFnZS4gVGhlIG5leHQgdGhyZWUgc3RhZ2VzIGluY2x1ZGUgc3VwcG9ydGl2ZSBhcmd1bWVudHMgb2YgdGhlIHRoZW9yeSBhbmQgdGhlIGxhc3QgdGhyZWUgYXJndW1lbnRzIGluY2x1ZGUgZGlzcHJvdmluZyBhcmd1bWVudHMgb2YgdGhlIHRoZW9yeS4gV2UgaHlwb3RoZXNpemVkIHRoYXQgdGhlIGdyb3VwIHdpdGggc2ltcGxlIGFyZ3VtZW50cyBzaG93cyBiZXR0ZXIgcGVyc3Vhc2lvbiAoYXMgaXQgcmVmbGVjdHMgaW4gaGlnaGVyIHJhdGluZ3MgZm9yIHRoZSBzdXBwb3J0aXZlIGFyZ3VtZW50cykgYW5kIGJldHRlciBkaXNzdWFzaW9uIChhcyBpdCByZWZsZWN0cyBpbiBsb3dlciByYXRpbmdzIGZvciB0aGUgb3Bwb3NpbmcgYXJndW1lbnRzKS4KCkluIHRoZSBsYXN0IHBhcnQgb2YgdGhlIHN0dWR5LCBwYXJ0aWNpcGFudHMgd2lsbCBiZSBhc2tlZCB0byBhbnN3ZXIgc2V2ZXJhbCBjb2duaXRpdmUgY2FwYWNpdHkvc3R5bGUgbWVhc3VyZXMgaW5jbHVkaW5nIENSVCwgQU9ULUUsIG1pbmR3YXJlLCBhbmQgbnVtZXJhY3kgc2NhbGVzLiBXZSBoeXBvdGhlc2l6ZWQgdGhhdCBjb2duaXRpdmUgYWJpbGl0eSwgY29nbml0aXZlIHN0eWxlLCBhbmQgb3Blbi1taW5kZWRuZXNzIGFyZSBwb3NpdGl2ZSBwcmVkaWN0b3JzIG9mIHBlcnN1YXNpb24gYW5kIGRpc3N1YXNpb24uIFRoZXNlIGFzc29jaWF0aW9ucyBzaG91bGQgYmUgbW9yZSBwcm9ub3VuY2VkIGZvciBwYXJ0aWNpcGFudHMgaW4gdGhlIGdyb3VwIHdpdGggY29tcGxleCBhcmd1bWVudHMgYmVjYXVzZSB0aGUgYWJpbGl0eSBhbmQgd2lsbGluZ25lc3MgdG8gZW5nYWdlIGluIGRlbGliZXJhdGl2ZSB0aGlua2luZyBtYXkgZmF2b3IgcGFydGljaXBhbnRzIHRvIGFzc2VzcyB0aGUgdW5kZXJseWluZyBsb2dpY2FsIHN0cnVjdHVyZSBvZiB0aG9zZSBhcmd1bWVudHMuIEhvd2V2ZXIsIGZvciBwYXJ0aWNpcGFudHMgaW4gdGhlIHNpbXBsZSBncm91cCwgdGhlIGxvZ2ljYWwgc3RydWN0dXJlIG9mIGFyZ3VtZW50cyBpcyBtb3JlIGV2aWRlbnQsIHNvIHBhcnRpY2lwYW50cyB3aXRoIGxvd2VyIGFiaWxpdHkgY2FuIHN0aWxsIGFzc2VzcyB0aGUgbG9naWNhbCBzdGF0dXMgb2YgdGhvc2UgYXJndW1lbnRzLgogCgpgYGB7ciBmaWcuYWxpZ249J2NlbnRlcicsIGVjaG89RkFMU0V9CmtuaXRyOjppbmNsdWRlX2dyYXBoaWNzKGhlcmUoJ2lucHV0cycsJ2V4cF9kZXNpZ24ucG5nJykpCmBgYAoKVGh1cywgb3VyIGh5cG90aGVzZXMgZm9yIHRoaXMgZXhwZXJpbWVudCBhcmUgYXMgZm9sbG93czoKCi0gUGFydGljaXBhbnRzIGluIHRoZSBncm91cCB3aXRoIHNpbXBsZSBhcmd1bWVudHMgaGF2ZSBoaWdoZXIgcmF0aW5ncyBmb3Igc3VwcG9ydGl2ZSBhcmd1bWVudHMgKFRoZXkgYXJlIG1vcmUgZWFzaWx5IHBlcnN1YWRlZCB0aGFuIHRob3NlIGluIHRoZSBncm91cCB3aXRoIGNvbXBsZXggYXJndW1lbnRzKS4KCi0gUGFydGljaXBhbnRzIGluIHRoZSBncm91cCB3aXRoIHNpbXBsZSBhcmd1bWVudHMgaGF2ZSBsb3dlciByYXRpbmdzIGZvciBvcHBvc2luZyBhcmd1bWVudHMgKFRoZXkgYXJlIG1vcmUgZWFzaWx5IGRpc3N1YWRlZCB0aGFuIHRob3NlIGluIHRoZSBncm91cCB3aXRoIGNvbXBsZXggYXJndW1lbnRzKS4KCi0gVGhlcmUgYXJlIHNpZ25pZmljYW50IGFzc29jaWF0aW9ucyBiZXR3ZWVuIENSVCwgQU9ULUUsIE51bWVyYWN5LCBhbmQgbWluZHdhcmUgd2l0aCBib3RoIHBlcnN1YXNpb24gYW5kIGRpc3N1YXNpb24gaW5kZXhlcyBpbiBlYWNoIGdyb3VwIGFuZCBpbiB0aGUgZW50aXJlIHNhbXBsZS4gVGhlIHJlbGF0aW9uc2hpcCBiZXR3ZWVuIHRoZXNlIG1lYXN1cmVzIHNob3VsZCBiZSBzdHJvbmdlciwgYWx0aG91Z2ggbm90IHNpZ25pZmljYW50bHksIGZvciBwYXJ0aWNpcGFudHMgaW4gdGhlIGdyb3VwIHdpdGggY29tcGxleCBhcmd1bWVudHMuCgoKYGBge3IgZWNobz1GQUxTRSwgb3V0LndpZHRoPSI1NTBweCIsIG91dC5oZWlnaHQ9IjQwMHB4In0Ka25pdHI6OmluY2x1ZGVfZ3JhcGhpY3MoaGVyZSgnaW5wdXRzJywncHJlZGljdGlvbl9wbG90LnBuZycpKQpgYGAKCgojIEdldHRpbmcgUmVhZHkKCkZpcnN0LCB3ZSBuZWVkIHRvIGRlc2lnbiB0aGUgZXhwZXJpbWVudC4gRm9yIHRoaXMgZXhwZXJpbWVudCwgd2UgdXNlIG9ubGluZSBwbGF0Zm9ybXMgZm9yIGRhdGEgY29sbGVjdGlvbi4gVGhlcmUgYXJlIHNldmVyYWwgb3B0aW9ucyBzdWNoIGFzIEdvcmlsbGEsIEpTcHN5Y2gsIFF1YWx0cmljcywgcHN5Y2hvSlMgKHBhdmxvdmlhKSwgZXRjLiBTaW5jZSB3ZSBkbyBub3QgbmVlZCBhbnkgcmVhY3Rpb24gdGltZSBkYXRhLCB3ZSBzaW1wbHkgdXNlIFF1YWx0cmljcy4gRm9yIGFuIG92ZXJ2aWV3IG9mIGRpZmZlcmVudCBsYWItYmFzZWQgYW5kIG9ubGluZSBwbGF0Zm9ybXMsIHNlZSBbaGVyZV0oaHR0cHM6Ly9vbWlkZ2hhc2VtaTIxLmdpdGh1Yi5pby9odW1hbl9kYXRhL1NjcmlwdHMvYmVoYXZpb3JhbF9kYXRhLmh0bWwpLiAKCk5leHQsIHdlIG5lZWQgdG8gZGVjaWRlIG9uIHRoZSBudW1iZXIgb2YgcGFydGljaXBhbnRzIChzYW1wbGUgc2l6ZSkuIEZvciB0aGlzIHN0dWR5LCB3ZSBkbyBub3Qgc3VlIHBvd2VyIGFuYWx5c2lzIHNpbmNlIHdlIGNhbm5vdCBhY2Nlc3MgbW9yZSB0aGFuIDEyMCBwYXJ0aWNpcGFudHMuIEhvd2V2ZXIsIGl0IGlzIGhpZ2hseSBzdWdnZXN0ZWQgY2FsY3VsYXRlIHNhbXBsZSBzaXplIHVzaW5nIHBvd2VyIGVzdGltYXRpb24uIFlvdSBjYW4gZmluZCBzb21lIG5pY2UgdHV0b3JpYWxzIG9uIGhvdyB0byBkbyB0aGF0IFtoZXJlXShodHRwczovL2p1bGlhbnF1YW5kdC5jb20vcG9zdC9wb3dlci1hbmFseXNpcy1ieS1kYXRhLXNpbXVsYXRpb24taW4tci1wYXJ0LWkvKSwgW2hlcmVdKGh0dHBzOi8vbmlja2NoLWsuZ2l0aHViLmlvL0Vjb25vbWV0cmljc1NsaWRlcy9XZWVrXzA4L1Bvd2VyX1NpbXVsYXRpb25zLmh0bWwpLCBhbmQgW2hlcmVdKGh0dHBzOi8vY3Jhbi5yLXByb2plY3Qub3JnL3dlYi9wYWNrYWdlcy9wYXJhbXRlc3QvdmlnbmV0dGVzL1NpbXVsYXRpbmctUG93ZXIuaHRtbCkuCgpBZnRlciB3ZSBjcmVhdGVkIHRoZSBleHBlcmltZW50IGFuZCBkZWNpZGVkIG9uIHRoZSBzYW1wbGUgc2l6ZSwgdGhlIG5leHQgc3RlcCBpcyB0byBwcmVyZXNpZ3RlciB0aGUgc3R1ZHkuIEhvd2V2ZXIsIGl0IHdvdWxkIGJlIGJldHRlciB0byBkbyBhIHBpbG90IHdpdGggNCBvciA1IHBhcnRpY2lwYW50cywgY2xlYW4gYWxsIHRoZSBkYXRhLCBkbyB0aGUgZGVzaXJlZCBhbmFseXNpcywgYW5kIHRoZW4gcHJlLXJlZ2lzdGVyIHRoZSBhbmFseXNpcyBhbmQgdGhvc2UgY29kZXMuIFlvdSBjYW4gZmluZCB0aGUgcHJlcmVnaXN0cmF0aW9uIGZvcm0gZm9yIHRoZSBjdXJyZW50IHN0dWR5IFtoZXJlXShodHRwczovL29zZi5pby83OXI2ZSkuCgpGaW5hbGx5LCB3ZSBuZWVkIHRvIHJlc3RydWN0dXJlIG91ciBwcm9qZWN0IGluIGEgdGlkeSBmb2xkZXIgd2l0aCBkaWZmZXJlbnQgc3ViLWZvbGRlcnMuIEhhdmluZyBhIGNsZWFuIGFuZCB0aWR5IGZvbGRlciBzdHJ1Y3R1cmUgY2FuIHNhdmUgdXMhIFRoZXJlIGFyZSBkaWZmZXJlbnQgZm9ybWF0cyBvZiBmb2xkZXIgc3RydWN0dXJlIChmb3IgZXhhbXBsZSwgc2VlIFtoZXJlXShodHRwOi8vbmlrb2xhLm1lL2ZvbGRlcl9zdHJ1Y3R1cmUuaHRtbCkgYW5kIFtoZXJlXShodHRwczovL3NsaWRlcy5jb20vZGpuYXZhcnJvL3dvcmtmbG93KSksIGJ1dCBmb3Igbm93LCB3ZSB1c2UgdGhlIGZvbGxvd2luZyBzdHJ1Y3R1cmU6CgpgYGB7ciBlY2hvPUZBTFNFLCBvdXQud2lkdGg9IjcwMHB4Iiwgb3V0LmhlaWdodD0iMjAwcHgifQprbml0cjo6aW5jbHVkZV9ncmFwaGljcyhoZXJlKCdpbnB1dHMnLCdmb2xkZXJfc3RydWN0dXJlLnBuZycpKQpgYGAKCgojIEludHJvZHVjdGlvbiB0byBSCmBgYHtyIG1lc3NhZ2U9RkFMU0UsIGV2YWw9Rn0KIyBsb2FkIGxpYnJhcmllcwpsaWJyYXJ5KHRpZHl2ZXJzZSkKbGlicmFyeShoZXJlKQpsaWJyYXJ5KGphbml0b3IpCmxpYnJhcnkoYnJvb20pCmxpYnJhcnkoYWZleCkKbGlicmFyeShlbW1lYW5zKQpsaWJyYXJ5KGtuaXRyKQpsaWJyYXJ5KGthYmxlRXh0cmEpCmxpYnJhcnkoZ2dzY2kpCmxpYnJhcnkocGF0Y2h3b3JrKQpsaWJyYXJ5KHNraW1yKQojIGluc3RhbGwucGFja2FnZXMoImRldnRvb2xzIikKIyBkZXZ0b29sczo6aW5zdGFsbF9naXRodWIoImVhc3lzdGF0cy9jb3JyZWxhdGlvbiIpCmxpYnJhcnkoImNvcnJlbGF0aW9uIikKb3B0aW9ucyhzY2lwZW49OTk5KSAjIHR1cm4gb2ZmIHNjaWVudGlmaWMgbm90YXRpb25zCm9wdGlvbnMoY29udHJhc3RzID0gYygnY29udHIuc3VtJywnY29udHIucG9seScpKSAjIHNldCB0aGUgY29udHJhc3Qgc3VtIGdsb2JhbGx5IApvcHRpb25zKGtuaXRyLmthYmxlLk5BID0gJycpCmBgYAoKUiBjYW4gYmUgdXNlZCBhcyBhIGNhbGN1bGF0b3IuIEZvciBtYXRoZW1hdGljYWwgcHVycG9zZXMsIGJlIGNhcmVmdWwgb2YgdGhlIG9yZGVyIGluIHdoaWNoIFIgZXhlY3V0ZXMgdGhlIGNvbW1hbmRzLgoKYGBge3J9CjEwICsgMTAKCjQgXiAyCgooMjUwIC8gNTAwKSAqIDEwMApgYGAKClIgaXMgYSBiaXQgZmxleGlibGUgd2l0aCBzcGFjaW5nIChidXQgbm8gc3BhY2luZyBpbiB0aGUgbmFtZSBvZiB2YXJpYWJsZXMgYW5kIHdvcmRzKQoKYGBge3J9CjEwKzEwCgoxMCAgICAgICAgICAgICAgICAgKyAgICAgICAgICAgMTAKYGBgCgpSIGNhbiBzb21ldGltZXMgdGVsbCB0aGF0IHlvdSdyZSBub3QgZmluaXNoZWQgeWV0CgpgYGB7ciBldmFsPUZ9CjEwICsKYGBgCgpIb3cgdG8gY3JlYXRlIGEgKnZhcmlhYmxlKj8gVmFyaWFibGUgYXNzaWdubWVudCB1c2luZyBgPC1gIGFuZCBgPWAuIE5vdGUgdGhhdCBSIGlzIGNhc2Ugc2Vuc2l0aXZlIGZvciBldmVyeXRoaW5nCgpgYGB7cn0KcGF5IDwtIDI1MAoKbW9udGggPSAxMgoKcGF5ICogbW9udGgKCnNhbGFyeSA8LSBwYXkgKiBtb250aApgYGAKCgpGZXcgcG9pbnRzIGluIG5hbWluZyB2YXJpYWJsZXMgYW5kIHZlY3RvcnM6IHVzZSBzaG9ydCwgaW5mb3JtYXRpdmUgd29yZHMsIGtlZXAgc2FtZSBtZXRob2QgKGUuZy4sIG5vdCB1c2luZyBjYXBpdGFsIHdvcmRzLCB1c2Ugb25seSBfIG9yIC4gKS4KCiMjIEZ1bmN0aW9uIApGdW5jdGlvbiBpcyBhIHNldCBvZiBzdGF0ZW1lbnRzIGNvbWJpbmVkIHRvZ2V0aGVyIHRvIHBlcmZvcm0gYSBzcGVjaWZpYyB0YXNrLiBXaGVuIHdlIHVzZSBhIGJsb2NrIG9mIGNvZGUgcmVwZWF0ZWRseSwgd2UgY2FuIGNvbnZlcnQgaXQgdG8gYSBmdW5jdGlvbi4gVG8gd3JpdGUgYSBmdW5jdGlvbiwgZmlyc3QsIHlvdSBuZWVkIHRvICpkZWZpbmUqIGl0OgoKYGBge3J9Cm15X211bHRpcGxpZXIgPC0gZnVuY3Rpb24oYSxiKXsKICByZXN1bHQgPSBhICogYgogIHJldHVybiAocmVzdWx0KQp9CmBgYAoKVGhpcyBjb2RlIGRvIG5vdGhpbmcuIFRvIGdldCBhIHJlc3VsdCwgeW91IG5lZWQgdG8gKmNhbGwqIGl0OgoKYGBge3J9Cm15X211bHRpcGxpZXIgKDIsNCkKYGBgCgpGb3J0dW5hdGVseSwgeW91IGRvIG5vdCBuZWVkIHRvIHdyaXRlIGV2ZXJ5dGhpbmcgZnJvbSBzY3JhdGNoLiBSIGhhcyBsb3RzIG9mIGJ1aWx0LWluIGZ1bmN0aW9ucyB0aGF0IHlvdSBjYW4gdXNlOgpgYGB7cn0Kcm91bmQoNTQuNjc4NykKcm91bmQoNTQuNTc4NywgZGlnaXRzID0gMikKYGBgCgpVc2UgYD9gIGJlZm9yZSB0aGUgZnVuY3Rpb24gbmFtZSB0byBnZXQgc29tZSBoZWxwLiBGb3IgZXhhbXBsZSwgYD9yb3VuZGAuIFlvdSB3aWxsIHNlZSBtYW55IGZ1bmN0aW9ucyBpbiB0aGUgcmVzdCBvZiB0aGUgd29ya3Nob3AuCgojIyBCYXNpYyBEYXRhIFR5cGVzIGluIFI6CgpmdW5jdGlvbiBgY2xhc3MoKWAgaXMgdXNlZCB0byBzaG93IHdoYXQgaXMgdGhlIHR5cGUgb2YgYSB2YXJpYWJsZS4KCgoxLiAqTG9naWNhbCo6IGBUUlVFYCwgYEZBTFNFYCBjYW4gYmUgYWJicmV2aWF0ZWQgYXMgYFRgLCBgRmAuICBUaGV5IGhhcyB0byBiZSBjYXBpdGFsLCAndHJ1ZScgaXMgbm90IGEgbG9naWNhbCBkYXRhOgpgYGB7cn0KY2xhc3MoVFJVRSkKY2xhc3MoRikKYGBgCgoyLiAqTnVtZXJpYyo6IGFsbCBudW1iZXJzIGUuZy4gNSwgIDEwLjUsICAxMSwzNzsgIGEgc3BlY2lhbCB0eXBlIG9mIG51bWVyaWMgaXMgImludGVnZXIiIHdoaWNoIGlzIG51bWJlcnMgd2l0aG91dCBkZWNpbWFsLiBJbnRlZ2VycyBhcmUgYWx3YXlzIG51bWVyaWMsIGJ1dCBudW1lcmljIGlzIG5vdCBhbHdheXMgaW50ZWdlcjoKYGBge3J9CmNsYXNzKDIpCmNsYXNzKDEzLjQ2KQpgYGAKCjMuICpDaGFyYWN0ZXIqOiB0ZXh0IGZvciBleGFtcGxlLCAiSSBsb3ZlIFIiIG9yICI0IiBvciAiNC41IjoKYGBge3J9CmNsYXNzKCJoYSBoYSBoYSBoYSIpCmNsYXNzKCI1Ni42IikKY2xhc3MoIlRSVUUiKQpgYGAKCkNhbiB3ZSBjaGFuZ2UgdGhlIHR5cGUgb2YgZGF0YSBpbiBhIHZhcmlhYmxlPyBZZXMsIHlvdSBuZWVkIHRvIHVzZSB0aGUgZnVuY3Rpb24gYGFzLi0tLSgpYAoKYGBge3J9CmFzLm51bWVyaWMoVFJVRSkKYXMuY2hhcmFjdGVyKDQpCmFzLm51bWVyaWMoIjQuNSIpCmFzLm51bWVyaWMoIkhlbGxvIikKYGBgCgoKIyMgRGF0YSBTdHJ1Y3R1cmVzIGluIFIKCgoqKlZlY3RvcioqOiB3aGVuIHRoZXJlIGFyZSBtb3JlIHRoYW4gb25lIG51bWJlciBvciBsZXR0ZXIgc3RvcmVkLiBVc2UgdGhlIGNvbWJpbmUgZnVuY3Rpb24gYygpIGZvciB0aGF0LgoKYGBge3J9CnNhbGUgPC0gYygxLCAyLCAzLDQsIDUsIDYsIDcsIDgsIDksIDEwKSAjIGFsc28gc2FsZSA8LSBjKDE6MTApCgpzYWxlIDwtIGMoMToxMCkKCnNhbGUgKiBzYWxlCmBgYAoKKlN1YnNldHRpbmcgYSB2ZWN0b3IqOgoKYGBge3J9CmRheXMgPC0gYygiU2F0dXJkYXkiLCAiU3VuZGF5IiwgIk1vbmRheSIsICJUdWVzZGF5IiwgIldlZG5lc2RheSIsICJUaHVyc2RheSIsICJGcmlkYXkiKQoKZGF5c1syXQpkYXlzWy0yXQoKZGF5c1tjKDIsIDMsIDQpXQpgYGAKCgojIyMgRXhlcmNpc2U6CgpDcmVhdGUgYSB2ZWN0b3IgbmFtZWQgYG15X3ZlY3RvcmAgd2l0aCBudW1iZXJzIGZyb20gMCB0byAxMDAwIGluIGl0OgoKYGBge3J9Cm15X3ZlY3RvciA8LSAoMDoxMDAwKQoKbWVhbihteV92ZWN0b3IpCm1lZGlhbihteV92ZWN0b3IpCm1pbihteV92ZWN0b3IpCnJhbmdlKG15X3ZlY3RvcikKY2xhc3MobXlfdmVjdG9yKQpzdW0obXlfdmVjdG9yKQpzZChteV92ZWN0b3IpCmBgYAoKKipMaXN0Kio6IGFsbG93cyB5b3UgdG8gZ2F0aGVyIGEgdmFyaWV0eSBvZiBvYmplY3RzIHVuZGVyIG9uZSBuYW1lICh0aGF0IGlzLCB0aGUgbmFtZSBvZiB0aGUgbGlzdCkgaW4gYW4gb3JkZXJlZCB3YXkuIFRoZXNlIG9iamVjdHMgY2FuIGJlIG1hdHJpY2VzLCB2ZWN0b3JzLCBkYXRhIGZyYW1lcywgZXZlbiBvdGhlciBsaXN0LgoKYGBge3J9Cm15X2xpc3QgPSBsaXN0KHNhbGUsIDEsIDMsIDQ6NywgIkhFTExPIiwgImhlbGxvIiwgRkFMU0UpCm15X2xpc3QKYGBgCgoqKkZhY3RvcioqOiBGYWN0b3JzIHN0b3JlIHRoZSB2ZWN0b3IgYWxvbmcgd2l0aCB0aGUgZGlzdGluY3QgdmFsdWVzIG9mIHRoZSBlbGVtZW50cyBpbiB0aGUgdmVjdG9yIGFzIGxhYmVscy4gVGhlIGxhYmVscyBhcmUgYWx3YXlzIGNoYXJhY3RlciBpcnJlc3BlY3RpdmUgb2Ygd2hldGhlciBpdCBpcyBudW1lcmljIG9yIGNoYXJhY3Rlci4gRm9yIGV4YW1wbGUsIHZhcmlhYmxlIGdlbmRlciB3aXRoICJtYWxlIiBhbmQgImZlbWFsZSIgZW50cmllczoKCmBgYHtyfQpnZW5kZXIgPC0gYygibWFsZSIsICJtYWxlIiwgIm1hbGUiLCAiIGZlbWFsZSIsICJmZW1hbGUiLCAiZmVtYWxlIikKZ2VuZGVyIDwtIGZhY3RvcihnZW5kZXIpCmBgYAoKUiBub3cgdHJlYXRzIGdlbmRlciBhcyBhIG5vbWluYWwgKGNhdGVnb3JpY2FsKSB2YXJpYWJsZTogMT1mZW1hbGUsIDI9bWFsZSBpbnRlcm5hbGx5IChhbHBoYWJldGljYWxseSkuCmBgYHtyfQpzdW1tYXJ5KGdlbmRlcikKYGBgCgoqUXVlc3Rpb24qOiB3aHkgd2hlbiB3ZSByYW4gdGhlIGFib3ZlIGZ1bmN0aW9uIGkuZS4gc3VtbWFyeSgpLCBpdCBzaG93ZWQgdGhyZWUgYW5kIG5vdCB0d28gbGV2ZWxzIG9mIHRoZSBkYXRhPyAqSGludCo6IHJ1biAnZ2VuZGVyJy4KCmBgYHtyfQpnZW5kZXIKYGBgCgpTbywgYmUgY2FyZWZ1bCBvZiBzcGFjZXMhCgojIyMgRXhlcmNpc2U6CkNyZWF0ZSBhIGdlbmRlciBmYWN0b3Igd2l0aCAzMCBtYWxlIGFuZCA0MCBmZW1hbGVzICgqSGludCo6IHVzZSB0aGUgYHJlcCgpYCBmdW5jdGlvbik6CmBgYHtyfQpnZW5kZXIgPC0gYyhyZXAoIm1hbGUiLDMwKSwgcmVwKCJmZW1hbGUiLCA0MCkpCmdlbmRlciA8LSBmYWN0b3IoZ2VuZGVyKQpnZW5kZXIKYGBgCgpUaGVyZSBhcmUgdHdvIHR5cGVzIG9mIGNhdGVnb3JpY2FsIHZhcmlhYmxlczogbm9taW5hbCBhbmQgb3JkaW5hbC4gSG93IHRvIGNyZWF0ZSBvcmRlcmVkIGZhY3RvcnMgKHdoZW4gdGhlIHZhcmlhYmxlIGlzIG5vbWluYWwgYW5kIHZhbHVlcyBjYW4gYmUgb3JkZXJlZCk/IFdlIHNob3VsZCBhZGQgdHdvIGFkZGl0aW9uYWwgYXJndW1lbnRzIHRvIHRoZSBgZmFjdG9yKClgIGZ1bmN0aW9uOiBgb3JkZXJlZCA9IFRSVUVgLCBhbmQgYGxldmVscyA9IGMoImxldmVsMSIsICJsZXZlbDIiKWAuIEZvciBleGFtcGxlLCB3ZSBoYXZlIGEgdmVjdG9yIHRoYXQgc2hvd3MgcGFydGljaXBhbnRzJyBlZHVjYXRpb24gbGV2ZWwuCgpgYGB7cn0KZWR1PC1jKDMsMiwzLDQsMSwyLDIsMyw0KQoKZWR1Y2F0aW9uPC1mYWN0b3IoZWR1LCBvcmRlcmVkID0gVFJVRSkKbGV2ZWxzKGVkdWNhdGlvbikgPC0gYygiUHJpbWFyeSBzY2hvb2wiLCJoaWdoIHNjaG9vbCIsIkNvbGxlZ2UiLCJVbmkgZ3JhZHVhdGVkIikKZWR1Y2F0aW9uCmBgYAoKIyMjIEV4ZXJjaXNlOgpXZSBoYXZlIGEgZmFjdG9yIHdpdGggYHBhdGllbnRgIGFuZCBgY29udHJvbGAgdmFsdWVzLiBIZXJlLCB0aGUgZmlyc3QgbGV2ZWwgaXMgY29udHJvbCBhbmQgdGhlIHNlY29uZCBsZXZlbCBpcyBwYXRpZW50LiBDaGFuZ2UgdGhlIG9yZGVyIG9mIGxldmVscywgc28gcGF0aWVudCB3b3VsZCBiZSB0aGUgZmlyc3QgbGV2ZWw6CgpgYGB7cn0KaGVhbHRoX3N0YXR1cyA8LSBmYWN0b3IoYyhyZXAoJ3BhdGllbnQnLDUpLHJlcCgnY29udHJvbCcsNSkpKQpoZWFsdGhfc3RhdHVzCgpoZWFsdGhfc3RhdHVzX3Jlb3JkZXJlZCA8LSBmYWN0b3IoaGVhbHRoX3N0YXR1cywgbGV2ZWxzID0gYygncGF0aWVudCcsJ2NvbnRyb2wnKSkKaGVhbHRoX3N0YXR1c19yZW9yZGVyZWQKYGBgCgpGaW5hbGx5LCBjYW4geW91IHJlbGFiZWwgYm90aCBsZXZlbHMgdG8gdXBwZXJjYXNlIGNoYXJhY3RlcnM/ICgqSGludCo6IGNoZWNrIGA/ZmFjdG9yYCkKCmBgYHtyfQpoZWFsdGhfc3RhdHVzX3JlbGFiZWxlZCA8LSBmYWN0b3IoaGVhbHRoX3N0YXR1cywgbGV2ZWxzID0gYygncGF0aWVudCcsJ2NvbnRyb2wnKSwgbGFiZWxzID0gYygnUGF0aWVudCcsJ0NvbnRyb2wnKSkKaGVhbHRoX3N0YXR1c19yZWxhYmVsZWQKYGBgCgoKKipNYXRyaWNlcyoqOiBBbGwgY29sdW1ucyBpbiBhIG1hdHJpeCBtdXN0IGhhdmUgdGhlIHNhbWUgbW9kZShudW1lcmljLCBjaGFyYWN0ZXIsIGV0Yy4pIGFuZCB0aGUgc2FtZSBsZW5ndGguIEl0IGNhbiBiZSBjcmVhdGVkIHVzaW5nIGEgdmVjdG9yIGlucHV0IHRvIHRoZSBtYXRyaXggZnVuY3Rpb24uCgpgYGB7cn0KbXlfbWF0cml4ID0gbWF0cml4KGMoMSwyLDMsNCw1LDYsNyw4LDkpLCBucm93ID0gMywgbmNvbCA9IDMpCgpteV9tYXRyaXgKYGBgCgoqKkRhdGEgZnJhbWVzKio6ICh0d28tZGltZW5zaW9uYWwgb2JqZWN0cykgY2FuIGhvbGQgbnVtZXJpYywgY2hhcmFjdGVyIG9yIGxvZ2ljYWwgdmFsdWVzLiBXaXRoaW4gYSBjb2x1bW4gYWxsIGVsZW1lbnRzIGhhdmUgdGhlIHNhbWUgZGF0YSB0eXBlLCBidXQgZGlmZmVyZW50IGNvbHVtbnMgY2FuIGJlIG9mIGRpZmZlcmVudCBkYXRhIHR5cGUuIExldCdzIGNyZWF0ZSBhIGRhdGFmcmFtZToKCmBgYHtyfQppZCA8LSAxOjIwMApncm91cCA8LSBjKHJlcCgiUHN5Y2hvdGhlcmFweSIsIDEwMCksIHJlcCgiTWVkaWNhdGlvbiIsIDEwMCkpCnJlc3BvbnNlIDwtIGMocm5vcm0oMTAwLCBtZWFuID0gMzAsIHNkID0gNSksCiAgICAgICAgICAgICBybm9ybSgxMDAsIG1lYW4gPSAyNSwgc2QgPSA1KSkKCm15X2RhdGFmcmFtZSA8LWRhdGEuZnJhbWUoUGF0aWVudCA9IGlkLAogICAgICAgICAgICAgICAgICAgICAgICAgIFRyZWF0bWVudCA9IGdyb3VwLAogICAgICAgICAgICAgICAgICAgICAgICAgIFJlc3BvbnNlID0gcmVzcG9uc2UpCmBgYAoKV2UgYWxzbyBjb3VsZCBoYXZlIGRvbmUgdGhlIGJlbG93CgpgYGB7cn0KbXlfZGF0YWZyYW1lIDwtZGF0YS5mcmFtZShQYXRpZW50ID0gYygxOjIwMCksCiAgICAgICAgICAgICAgICAgICAgICAgICAgVHJlYXRtZW50ID0gYyhyZXAoIlBzeWNob3RoZXJhcHkiLCAxMDApLCByZXAoIk1lZGljYXRpb24iLCAxMDApKSwKICAgICAgICAgICAgICAgICAgICAgICAgICBSZXNwb25zZSA9IGMocm5vcm0oMTAwLCBtZWFuID0gMzAsIHNkID0gNSksCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHJub3JtKDEwMCwgbWVhbiA9IDI1LCBzZCA9IDUpKSkKYGBgCgpJbiBsYXJnZSBkYXRhIHNldHMsIHRoZSBmdW5jdGlvbiBoZWFkKCkgZW5hYmxlcyB5b3UgdG8gc2hvdyB0aGUgZmlyc3Qgb2JzZXJ2YXRpb25zIG9mIGEgZGF0YSBmcmFtZXMuIFNpbWlsYXJseSwgdGhlIGZ1bmN0aW9uIHRhaWwoKSBwcmludHMgb3V0IHRoZSBsYXN0IG9ic2VydmF0aW9ucyBpbiB5b3VyIGRhdGEgc2V0LgoKYGBge3IgZXZhbD1GfQpoZWFkKG15X2RhdGFmcmFtZSkgCnRhaWwobXlfZGF0YWZyYW1lKQpgYGAKCmBgYHtyIGVjaG89Rn0KaGVhZChteV9kYXRhZnJhbWUpICU+JQogIGtuaXRyOjprYWJsZSgpICU+JQogIGthYmxlX3N0eWxpbmcoYm9vdHN0cmFwX29wdGlvbnMgPSBjKCJzdHJpcGVkIiwgImJvcmRlcmVkIiwgImNvbmRlbnNlZCIpLCBmaXhlZF90aGVhZCA9IFQsIGZ1bGxfd2lkdGggPSBUKQp0YWlsKG15X2RhdGFmcmFtZSklPiUKICBrbml0cjo6a2FibGUoKSAlPiUKICBrYWJsZV9zdHlsaW5nKGJvb3RzdHJhcF9vcHRpb25zID0gYygic3RyaXBlZCIsICJib3JkZXJlZCIsICJjb25kZW5zZWQiKSwgZml4ZWRfdGhlYWQgPSBULCBmdWxsX3dpZHRoID0gVCkKYGBgCgpTaW1pbGFyIHRvIHZlY3RvcnMgYW5kIG1hdHJpY2VzLCBicmFja2V0cyBbXSBhcmUgdXNlZCB0byBzZWxlY3RzIGRhdGEgZnJvbSByb3dzIGFuZCBjb2x1bW5zIGluIGRhdGEuZnJhbWVzOgoKYGBge3J9Cm15X2RhdGFmcmFtZVszNSwgM10KYGBgCgojIyMgRXhlcmNpc2UKCkhvdyBjYW4gd2UgZ2V0IGFsbCBjb2x1bW5zLCBidXQgb25seSBmb3IgdGhlIGZpcnN0IDEwIHBhcnRpY2lwYW50cz8KCmBgYHtyIGV2YWw9Rn0KbXlfZGF0YWZyYW1lWzE6MTAsIF0KYGBgCgpgYGB7ciBlY2hvPUZ9CmtuaXRyOjprYWJsZShteV9kYXRhZnJhbWVbMToxMCwgXSkgJT4lCiAga2FibGVfc3R5bGluZyhib290c3RyYXBfb3B0aW9ucyA9IGMoInN0cmlwZWQiLCAiYm9yZGVyZWQiLCAiY29uZGVuc2VkIiksIGZpeGVkX3RoZWFkID0gVCwgZnVsbF93aWR0aCA9IFQpCgpgYGAKSG93IHRvIGdldCBvbmx5IHRoZSBSZXNwb25zZSBjb2x1bW4gZm9yIGFsbCBwYXJ0aWNpcGFudHM/CgpgYGB7cn0KbXlfZGF0YWZyYW1lWyAsIDNdCmBgYAoKQW5vdGhlciBlYXNpZXIgd2F5IGZvciBzZWxlY3RpbmcgcGFydGljdWxhciBpdGVtcyBpcyB1c2luZyB0aGVpciBuYW1lcyB0aGF0IGlzIG1vcmUgaGVscGZ1bCB0aGFuIG51bWJlciBvZiB0aGUgcm93cyBpbiBsYXJnZSBkYXRhIHNldHM6CmBgYHtyIGV2YWw9Rn0KbXlfZGF0YWZyYW1lWyAsICJSZXNwb25zZSJdCiMgT1I6Cm15X2RhdGFmcmFtZSRSZXNwb25zZQoKYGBgCgoKIyBEYXRhIENsZWFuaW5nCgpOb3csIHN1cHBvc2Ugd2UgdGVzdGVkIDE0MSBzdHVkZW50cy4gRmlyc3QsIGxldCdzIHJlYWQgYW5kIGNoZWNrIHRoZSB1bmNsZWFuZWQgZGF0YToKYGBge3IgbWVzc2FnZT1GLCB3YXJuaW5nPUYsIGV2YWw9Rn0KIyByZWFkIHRoZSByYXcgZGF0YQpyYXdfZGF0YSA8LSByZWFkX2NzdihoZXJlKCJyYXdfZGF0YSIsInJhd19hcmd1bWVudGF0aXZlX2V4cDEuY3N2IikpCmhlYWQocmF3X2RhdGEpCmBgYAoKYGBge3IgbWVzc2FnZT1GLCB3YXJuaW5nPUYsIGVjaG89Rn0KIyByZWFkIHRoZSByYXcgZGF0YQpyYXdfZGF0YSA8LSByZWFkX2NzdihoZXJlKCJyYXdfZGF0YSIsInJhd19hcmd1bWVudGF0aXZlX2V4cDEuY3N2IikpCgprbml0cjo6a2FibGUoaGVhZChyYXdfZGF0YSkpICU+JQogIGthYmxlX3N0eWxpbmcoYm9vdHN0cmFwX29wdGlvbnMgPSBjKCJzdHJpcGVkIiwgImJvcmRlcmVkIiwgImNvbmRlbnNlZCIpLCBmaXhlZF90aGVhZCA9IFQsIGZ1bGxfd2lkdGggPSBGKSU+JQogIHNjcm9sbF9ib3god2lkdGggPSAiNzgwcHgiKQpgYGAKCk5vdywgbGV0J3MgZG8gc29tZSBjbGVhbmluaW5nIHVzaW5nIGBkcGx5cmAsIGB0aWR5cmAgYW5kIG90aGVyIGB0aWR5dmVyc2VgIGxpYnJhcmllcy4gRmluYWxseSwgd2Ugd2lsbCBjaGVjayB0aGUgZGF0YToKYGBge3IgbWVzc2FnZT1GLCB3YXJuaW5nPUYsIGV2YWw9Rn0KY2xlYW5lZF9kYXRhIDwtIHJhd19kYXRhICU+JSAKICBmaWx0ZXIocHJvZ3Jlc3MgPT0gMTAwKSAlPiUgIyBmaWx0ZXIgb3V0IHVuZmluaXNoZWQgcGFydGljaXBhbnRzCiAgc2VsZWN0KC1lbmRfZGF0ZSwgLXN0YXR1cywtaXBfYWRkcmVzcywgLWR1cmF0aW9uX2luX3NlY29uZHMsIC1yZWNvcmRlZF9kYXRlOi11c2VyX2xhbmd1YWdlKSAlPiUgI3JlbW92ZSBzb21lIHVzZWxlc3MgY29sdW1ucwogIG11dGF0ZShhb3RlX3RvdGFsPSBhb3RlMSthb3RlMithb3RlMythb3RlNCthb3RlNSthb3RlNithb3RlNythb3RlOCwgIyBjcmVhdGUgYSB0b3RhbCBzY29yZSBmb3Igb3VyIHF1ZXN0aW9ubmFpcmUKICAgICAgICAgYXR0ZW50aXZlPSBjYXNlX3doZW4oYXR0ZW50aW9uX2NvcnJlY3QgPj0gMn4gJ1llcycsVH4gJ05vJykpICU+JQogIG11dGF0ZShjcnQxPSBjYXNlX3doZW4oY3J0MT09JzQnfiAxLFR+MCksCiAgICAgICAgIGNydDI9IGNhc2Vfd2hlbihjcnQyPT0nMTAnfiAxLFR+MCksCiAgICAgICAgIGNydDM9IGNhc2Vfd2hlbihjcnQzPT0nMzknfiAxLFR+MCksCiAgICAgICAgIGNydF90b3RhbD0gY3J0MSArIGNydDIgKyBjcnQzKSAlPiUKICBzZWxlY3QoLWF0dGVudGlvbjE6LWFvdGU4KSAlPiUKICBwaXZvdF9sb25nZXIoY29scyA9IGMoc3RhZ2UxX3NpbXBsZTpzdGFnZTdfc2ltcGxlLHN0YWdlMV9jb21wbGV4OnN0YWdlN19jb21wbGV4KSxuYW1lc190byA9ICdzdGFnZScsdmFsdWVzX3RvID0gJ3RydXRoX2VzdGltYXRlJykgJT4lICMgbWFrZSBvdXIgZGF0YWZyYW1lIGxvbmcKICAjcGl2b3Rfd2lkZXIobmFtZXNfZnJvbSA9IHN0YWdlLCB2YWx1ZXNfZnJvbT0gdHJ1dGhfZXN0aW1hdGUpICMgdGhpcyBjb2RlIGNoYW5nZSBvdXIgZGF0YWZyYW1lIGJhY2sgdG8gd2lkZQogIGZpbHRlcighaXMubmEodHJ1dGhfZXN0aW1hdGUpKSAlPiUgI3JlbW92ZSByb3dzIHdpdGggdHJ1dGhfZXN0aW1hdGUgPT0gTkEKICBtdXRhdGUoc3RhZ2U9IGdzdWIoIl8uKiIsICIiLCBzdGFnZSkpICU+JQogIHJlbmFtZShjb25zZW50PSBjb25zZW50X2Zvcm0pICU+JSAjIHJlbmFtZSBhIGNvbHVtbgogICNtdXRhdGVfaWYoaXMuY2hhcmFjdGVyLCBmYWN0b3IpICU+JQogIG11dGF0ZShzdWJqZWN0PSBmYWN0b3Ioc3ViamVjdCksICMgY29udmVydCBhbGwgY2hhcmFjdGVycyB0byBmYWN0b3IKICAgICAgICAgZ3JvdXAgPSBmYWN0b3IoZ3JvdXApLAogICAgICAgICBhdHRlbnRpdmUgPSBmYWN0b3IoYXR0ZW50aXZlKSwKICAgICAgICAgc3RhZ2UgPSBmYWN0b3Ioc3RhZ2UpKQpgYGAKCgpgYGB7ciBtZXNzYWdlPUYsIHdhcm5pbmc9RiwgZWNobz1GfQpjbGVhbmVkX2RhdGEgPC0gcmF3X2RhdGEgJT4lIAogIGZpbHRlcihwcm9ncmVzcyA9PSAxMDApICU+JSAjIGZpbHRlciBvdXQgdW5maW5pc2hlZCBwYXJ0aWNpcGFudHMKICBzZWxlY3QoLWVuZF9kYXRlLCAtc3RhdHVzLC1pcF9hZGRyZXNzLCAtZHVyYXRpb25faW5fc2Vjb25kcywgLXJlY29yZGVkX2RhdGU6LXVzZXJfbGFuZ3VhZ2UpICU+JSAjcmVtb3ZlIHNvbWUgdXNlbGVzcyBjb2x1bW5zCiAgbXV0YXRlKGFvdGVfdG90YWw9IGFvdGUxK2FvdGUyK2FvdGUzK2FvdGU0K2FvdGU1K2FvdGU2K2FvdGU3K2FvdGU4LCAjIGNyZWF0ZSBhIHRvdGFsIHNjb3JlIGZvciBvdXIgcXVlc3Rpb25uYWlyZQogICAgICAgICBhdHRlbnRpdmU9IGNhc2Vfd2hlbihhdHRlbnRpb25fY29ycmVjdCA+PSAyfiAnWWVzJyxUfiAnTm8nKSkgJT4lCiAgbXV0YXRlKGNydDE9IGNhc2Vfd2hlbihjcnQxPT0nNCd+IDEsVH4wKSwKICAgICAgICAgY3J0Mj0gY2FzZV93aGVuKGNydDI9PScxMCd+IDEsVH4wKSwKICAgICAgICAgY3J0Mz0gY2FzZV93aGVuKGNydDM9PSczOSd+IDEsVH4wKSwKICAgICAgICAgY3J0X3RvdGFsPSBjcnQxICsgY3J0MiArIGNydDMpICU+JQogIHNlbGVjdCgtYXR0ZW50aW9uMTotYW90ZTgpICU+JQogIHBpdm90X2xvbmdlcihjb2xzID0gYyhzdGFnZTFfc2ltcGxlOnN0YWdlN19zaW1wbGUsc3RhZ2UxX2NvbXBsZXg6c3RhZ2U3X2NvbXBsZXgpLG5hbWVzX3RvID0gJ3N0YWdlJyx2YWx1ZXNfdG8gPSAndHJ1dGhfZXN0aW1hdGUnKSAlPiUgIyBtYWtlIG91ciBkYXRhZnJhbWUgbG9uZwogICNwaXZvdF93aWRlcihuYW1lc19mcm9tID0gc3RhZ2UsIHZhbHVlc19mcm9tPSB0cnV0aF9lc3RpbWF0ZSkgIyB0aGlzIGNvZGUgY2hhbmdlIG91ciBkYXRhZnJhbWUgYmFjayB0byB3aWRlCiAgZmlsdGVyKCFpcy5uYSh0cnV0aF9lc3RpbWF0ZSkpICU+JSAjcmVtb3ZlIHJvd3Mgd2l0aCB0cnV0aF9lc3RpbWF0ZSA9PSBOQQogIG11dGF0ZShzdGFnZT0gZ3N1YigiXy4qIiwgIiIsIHN0YWdlKSkgJT4lCiAgcmVuYW1lKGNvbnNlbnQ9IGNvbnNlbnRfZm9ybSkgJT4lICMgcmVuYW1lIGEgY29sdW1uCiAgI211dGF0ZV9pZihpcy5jaGFyYWN0ZXIsIGZhY3RvcikgJT4lCiAgbXV0YXRlKHN1YmplY3Q9IGZhY3RvcihzdWJqZWN0KSwgIyBjb252ZXJ0IGFsbCBjaGFyYWN0ZXJzIHRvIGZhY3RvcgogICAgICAgICBncm91cCA9IGZhY3Rvcihncm91cCksCiAgICAgICAgIGF0dGVudGl2ZSA9IGZhY3RvcihhdHRlbnRpdmUpLAogICAgICAgICBzdGFnZSA9IGZhY3RvcihzdGFnZSkpCgprbml0cjo6a2FibGUoaGVhZChjbGVhbmVkX2RhdGEpKSAlPiUKICBrYWJsZV9zdHlsaW5nKGJvb3RzdHJhcF9vcHRpb25zID0gYygic3RyaXBlZCIsICJib3JkZXJlZCIsICJjb25kZW5zZWQiKSwgZml4ZWRfdGhlYWQgPSBULCBmdWxsX3dpZHRoID0gRiklPiUKICBzY3JvbGxfYm94KHdpZHRoID0gIjc4MHB4IikKYGBgCgpPaywgbm93IHRoZSBkYXRhIGlzIGNsZWFuIGFuZCB0aWR5IHdoaWNoIG1lYW5zOgoKPiAxLiBFYWNoIHZhcmlhYmxlIGZvcm1zIGEgY29sdW1uLgoyLiBFYWNoIG9ic2VydmF0aW9uIGZvcm1zIGEgcm93LgozLiBFYWNoIHR5cGUgb2Ygb2JzZXJ2YXRpb25hbCB1bml0IGZvcm1zIGEgdGFibGUgKFtXaWNraGFtXShodHRwczovL3ZpdGEuaGFkLmNvLm56L3BhcGVycy90aWR5LWRhdGEucGRmKSwgMjAxNCkuCgpDaGVjayB0aGUgZGF0YWZyYW1lIGFuZCBhbGwgdGhlIGRhdGEgdHlwZXM6CmBgYHtyfQpzdHIoY2xlYW5lZF9kYXRhKQpgYGAKCkZpbmFsbHksIHdlIHNhdmUgb3VyIGRhdGEgdG8gdGhlIGBjbGVhbmVkX2RhdGFgIGZvbGRlci4KCmBgYHtyfQp3cml0ZV9jc3YoY2xlYW5lZF9kYXRhLCBoZXJlKCJjbGVhbmVkX2RhdGEiLCJhcmd1bWVudGF0aXZlX2V4cDEuY3N2IikpCmBgYAoKIyBEZXNjcmlwdGl2ZSBTdGF0aXN0aWNzCgo+IE5vdGU6IEFsbCB0aGUgZGF0YSB0aGF0IHdlIHVzZSBoZXJlIGlzIG1hbmlwdWxhdGVkIChmYWJyaWNhdGVkKSBmb3IgdGVhY2hpbmcgcHVycHVzZXMuIEluIG91ciBzdHVkeSwgd2UgZmFpbGVkIHRvIGZpbmQgc3VjaCBiZWF1dGlmdWwgYW5kIGludGVyZXN0aW5nIHJlc3VsdHMuCgpOb3csIGxldCdzIGRvIHNvbWUgZGVzY3JpcHRpdmUgc3RhdGlzdGljcy4gRmlyc3QsIHdlIGNhbiBvcGVuIGEgbmV3IHNjcmlwdCBjYWxsZWQgYGFuYWx5c2lzX2V4cDEucmAgYW5kIHJlYWQgdGhlIGNsZWFuZWQgZGF0YSBhZ2Fpbi4gCgpgYGB7ciBtZXNzYWdlPUYsIHdhcm5pbmc9Rix9CmRhdGFfZXhwMSA8LSByZWFkX2NzdihoZXJlKCJjbGVhbmVkX2RhdGEiLCJhcmd1bWVudGF0aXZlX2V4cDEuY3N2IikpCmBgYAoKSG93IG1hbnkgcGFydGljaXBhbnRzIGluIHRvdGFsPwoKYGBge3IgbWVzc2FnZT1GLCB3YXJuaW5nPUYsIGV2YWw9Rn0KZGF0YV9leHAxICU+JSBzdW1tYXJpc2Uobj0gbl9kaXN0aW5jdChzdWJqZWN0KSkKYGBgCgoKYGBge3IgbWVzc2FnZT1GLCB3YXJuaW5nPUYsIGVjaG89Rn0KZGF0YV9leHAxICU+JSBzdW1tYXJpc2Uobj0gbl9kaXN0aW5jdChzdWJqZWN0KSklPiUKICBrbml0cjo6a2FibGUoKSAlPiUKICBrYWJsZV9zdHlsaW5nKGJvb3RzdHJhcF9vcHRpb25zID0gYygic3RyaXBlZCIsICJib3JkZXJlZCIsICJjb25kZW5zZWQiKSwgZnVsbF93aWR0aCA9IEYpCmBgYAoKaG93IG1hbnkgcGFydGljaXBhbnRzIGluIGVhY2ggZ3JvdXA/CmBgYHtyIG1lc3NhZ2U9Riwgd2FybmluZz1GLCBldmFsPUZ9CmRhdGFfZXhwMSAlPiUgCiAgZ3JvdXBfYnkoc3ViamVjdCkgJT4lIAogIGZpbHRlcihyb3dfbnVtYmVyKCk9PTEpICU+JSAKICB1bmdyb3VwICgpICU+JSAKICBncm91cF9ieShncm91cCkgJT4lIAogIGNvdW50KCkgCmBgYAoKYGBge3IgbWVzc2FnZT1GLCB3YXJuaW5nPUYsIGVjaG89Rn0KZGF0YV9leHAxICU+JSBncm91cF9ieShzdWJqZWN0KSAlPiUgZmlsdGVyKHJvd19udW1iZXIoKT09MSkgJT4lIHVuZ3JvdXAgKCkgJT4lIGdyb3VwX2J5KGdyb3VwKSAlPiUgY291bnQoKSAlPiUKICBrbml0cjo6a2FibGUoKSAlPiUKICBrYWJsZV9zdHlsaW5nKGJvb3RzdHJhcF9vcHRpb25zID0gYygic3RyaXBlZCIsICJib3JkZXJlZCIsICJjb25kZW5zZWQiKSwgZml4ZWRfdGhlYWQgPSBUKQpgYGAKCkZpbmQgdGhlIG1lYW4gYW5kIHNkIGZvciBudW1lcmljIHZhcmlhYmxlcyB1c2luZyBiYXNlIFIgYHN1bW1hcnlgIGZ1bmN0aW9uOgoKYGBge3J9CmRhdGFfZXhwMSAlPiUgCiAgZ3JvdXBfYnkoc3ViamVjdCkgJT4lIAogIGZpbHRlcihyb3dfbnVtYmVyKCk9PTEpICU+JSAKICB1bmdyb3VwICgpICU+JQogIHN1bW1hcnkoKQpgYGAKCkFsdGVybmF0aXZlbHksIHdlIGNhbiB1c2UgYGJhc2UgUiBgc3VtbWFyeWAgZnVuY3Rpb25gc2tpbXJgIGxpYnJhcnk6CmBgYHtyIGV2YWw9Rn0KZGF0YV9leHAxICU+JSAKICBncm91cF9ieShzdWJqZWN0KSAlPiUgCiAgZmlsdGVyKHJvd19udW1iZXIoKT09MSkgJT4lIAogIHVuZ3JvdXAgKCkgJT4lIAogIGRwbHlyOjpzZWxlY3QgKGFnZSwgbnVtZXJhY3lfdG90YWwsIG1pbmR3YXJlX3RvdGFsLCBhb3RlX3RvdGFsLCBjcnRfdG90YWwpICU+JSAKICBza2ltcjo6c2tpbSgpCmBgYAoKYGBge3IgZWNobz1GfQpkYXRhX2V4cDEgJT4lIAogIGdyb3VwX2J5KHN1YmplY3QpICU+JSAKICBmaWx0ZXIocm93X251bWJlcigpPT0xKSAlPiUgCiAgdW5ncm91cCAoKSAlPiUgCiAgZHBseXI6OnNlbGVjdCAoYWdlLCBudW1lcmFjeV90b3RhbCwgbWluZHdhcmVfdG90YWwsIGFvdGVfdG90YWwsIGNydF90b3RhbCkgJT4lIAogIHNraW1yOjpza2ltKCkgJT4lCiAga25pdHI6OmthYmxlKCkgJT4lCiAga2FibGVfc3R5bGluZyhib290c3RyYXBfb3B0aW9ucyA9IGMoInN0cmlwZWQiLCAiYm9yZGVyZWQiLCAiY29uZGVuc2VkIiksIGZpeGVkX3RoZWFkID0gVCwgZnVsbF93aWR0aCA9IEYpJT4lCiAgc2Nyb2xsX2JveCh3aWR0aCA9ICI3ODBweCIpCmBgYAoKCiMjIyBFeGVyY2lzZQoKRm9yIHRoaXMgZXhlcmNpc2UsIHdlIHVzZSBhIGRhdGFzZXQgb2Ygb25lIG9mIG15IG93biBzdHVkaWVzLiBJbiB0aGlzIHN0dWR5LCB3ZSBhc2tlZCBwYXJ0aWNpcGFudHMgdG8gZ3Vlc3MgdGhlIHBoeXNpY2FsIGJyaWdodG5lc3Mgb2YgcmVhc29uaW5nIGFyZ3VtZW50cyBhbmQgdGhlbiB3ZSBnYXZlIGEgY29nbml0aXZlIGFiaWxpdHkgdGVzdC4gKFNlZSB0aGUgb3JpZ2luYWwgc3R1ZHkgW2hlcmVdKGh0dHBzOi8vb3NmLmlvL2VieG5mLykpLiBPcGVuIGBnaGFzZW1pX2JyaWdodG5lc3NfZXhwNC5jc3ZgIGZpbGUgYW5kIGFuc3dlciB0byB0aGUgZm9sbG93aW5nIHF1ZXN0aW9uczoKCjEuIEhvdyBtYW55IHBhcnRpY2lwYW50cyBkaWQgd2UgdGVzdCBpbiB0b3RhbD8KMi4gRmluZCBvdXQgaG93IG1hbnkgbWFsZSBhbmQgZmVtYWxlIHdlIHRlc3RlZC4KMy4gQ2FsY3VsYXRlIG1lYW4gYW5kIHNkIGZvciBhZ2UgYW5kIGNvZ25pdGl2ZSBhYmlsaXR5IChgYWg0YCkuCgoKYGBge3Igd2FybmluZz1GLCBtZXNzYWdlPUZ9CmdoYXNlbWlfZGF0YSA8LSByZWFkX2NzdihoZXJlKCJjbGVhbmVkX2RhdGEiLCJnaGFzZW1pX2JyaWdodG5lc3NfZXhwNC5jc3YiKSkKCmdoYXNlbWlfZGF0YSAlPiUgc3VtbWFyaXNlKG4gPSBuX2Rpc3RpbmN0KHBhcnRpY2lwYW50KSkgIyBudW1iZXIgb2YgcGFydGljaXBhbnRzOjIwMAoKZ2hhc2VtaV9kYXRhICU+JSBncm91cF9ieSAocGFydGljaXBhbnQpICU+JSBmaWx0ZXIgKHJvd19udW1iZXIoKT09MSkgJT4lIGdyb3VwX2J5IChnZW5kZXIpICU+JSBzdW1tYXJpc2Uobj0gbigpKSAlPiUgdW5ncm91cCgpICMgMTgzIGZlbWFsZSwgMTcgbWFsZQoKZ2hhc2VtaV9kYXRhICU+JSBkcGx5cjo6c2VsZWN0IChhZ2UsIGFoNCkgJT4lIHNraW1yOjpza2ltKCkgIyBtZWFuIGFuZCBzZCBmb3IgYWdlIGFuZCBjb2duaXRpdmUgYWJpbGl0eQpgYGAKCgojIERhdGEgVmlzdWFsaXphdGlvbgoKRmlyc3QsIHdlIG5lZWQgdG8gY3JlYXRlIGEgZGF0YXNldCB3aXRoIGFnZ3JlZ2F0ZWQgYHRydXRoIGVzdGltYXRlYCBzY29yZXMgb3ZlciBgZ3JvdXBgIGFuZCBgc3RhZ2VgLiBXZSB3aWxsIHVzZSB0aGlzIGRhdGFzZXQgZm9yIGxpbmUgYW5kIGJhciBncmFwaHMuCgpgYGB7ciBtZXNzYWdlPUYsIHdhcm5pbmc9RiwgZHBpPSAzMDAsIGZpZy5oZWlnaHQ9MywgZmlnLndpZHRoPTV9CgphZ2dyZWdhdGVkX2RhdGFfZXhwMSA8LSBkYXRhX2V4cDEgJT4lCiAgZ3JvdXBfYnkoc3RhZ2UsIGdyb3VwKSAlPiUKICBtdXRhdGUodHJ1dGhfZXN0aW1hdGUgPSBtZWFuKHRydXRoX2VzdGltYXRlKSkgJT4lCiAgdW5ncm91cCgpCgpiYXJwbG90X2V4cDEgPC0gYWdncmVnYXRlZF9kYXRhX2V4cDEgJT4lCiAgZ2dwbG90KGFlcyh4PXN0YWdlLCB5PSB0cnV0aF9lc3RpbWF0ZSwgZmlsbD1ncm91cCkpICsKICBnZW9tX2JhcihzdGF0ID0gImlkZW50aXR5IiwgcG9zaXRpb249ICJkb2RnZSIpKwogICMgc3RhdF9zdW1tYXJ5KGZ1bj0gbWVhbiwgZ2VvbSA9ICJiYXIiLCBwb3NpdGlvbiA9ICJkb2RnZSIpKyAjIGNhbiBiZSB1c2VkIGluc3RlYWQgb2YgZ2VvbV9iYXIoKSBmb3IgbG9uZyBkYXRhZnJhbWVzCiAgbGFicyAoeD0gJycsIHk9ICJUcnV0aCBMaWtlbGloaG9kIEVzdGltYXRlIikgKyAKICB0aGVtZV9idygpICsgCiAgc2NhbGVfZmlsbF9qYW1hKCkgCgpiYXJwbG90X2V4cDEKCgpiYXJwbG90X2ZhY2V0X2V4cDEgPC0gYWdncmVnYXRlZF9kYXRhX2V4cDEgJT4lCiAgZ2dwbG90KGFlcyh4PWdyb3VwLCB5PSB0cnV0aF9lc3RpbWF0ZSwgZmlsbD1zdGFnZSkpICsKICBnZW9tX2JhcihzdGF0ID0gImlkZW50aXR5IiwgcG9zaXRpb249ICJkb2RnZSIpKwogIGxhYnMgKHg9ICcnLCB5PSAiVHJ1dGggTGlrZWxpaGhvZCBFc3RpbWF0ZSIpICsgCiAgdGhlbWVfYncoKSArIAogIHRoZW1lKGxlZ2VuZC5wb3NpdGlvbiA9ICJub25lIiwKICAgICAgICBheGlzLnRleHQ9ZWxlbWVudF90ZXh0KHNpemU9MTEpLAogICAgICAgIGF4aXMudGl0bGUgPSBlbGVtZW50X3RleHQoc2l6ZSA9IDEyKSkgKwogIGZhY2V0X3dyYXAofnN0YWdlKSsKICBzY2FsZV9maWxsX2pjbygpIAoKYmFycGxvdF9mYWNldF9leHAxCgoKbGluZXBsb3RfZXhwMSA8LSBhZ2dyZWdhdGVkX2RhdGFfZXhwMSAlPiUKICBnZ3Bsb3QoYWVzKHg9ZmFjdG9yKHN0YWdlKSwgeT0gdHJ1dGhfZXN0aW1hdGUsIGdyb3VwPSBncm91cCwgY29sb3I9IGdyb3VwKSkgKwogIGdlb21fbGluZShhZXMobGluZXR5cGU9IGdyb3VwKSkgKwogIGdlb21fcG9pbnQoc2l6ZT0gNSkrCiAgbGFicyAoeD0gJycsIHk9ICJUcnV0aCBMaWtlbGloaG9kIEVzdGltYXRlIikgKyAKICB0aGVtZV9jbGFzc2ljKCkgKwogIHRoZW1lKGxlZ2VuZC5wb3NpdGlvbiA9ICJib3R0b20iLAogICAgICAgIGF4aXMudGV4dD1lbGVtZW50X3RleHQoc2l6ZT0xMSksCiAgICAgICAgYXhpcy50aXRsZSA9IGVsZW1lbnRfdGV4dChzaXplID0gMTIpKSArCiAgc2NhbGVfY29sb3JfbmVqbSgpIAoKbGluZXBsb3RfZXhwMQoKCnZpb2xpbnBsb3RfZXhwMSA8LSBkYXRhX2V4cDEgJT4lCiAgZ2dwbG90KGFlcyh4PWZhY3RvcihzdGFnZSksIHk9IHRydXRoX2VzdGltYXRlLCBmaWxsPSBncm91cCkpICsKICBnZW9tX3Zpb2xpbigpKwogIGxhYnMgKHg9ICcnLCB5PSAiVHJ1dGggTGlrZWxpaGhvZCBFc3RpbWF0ZSIpICsgCiAgdGhlbWVfYncoKSArIAogIHRoZW1lKGxlZ2VuZC5wb3NpdGlvbiA9ICJib3R0b20iLAogICAgICAgIGF4aXMudGV4dD1lbGVtZW50X3RleHQoc2l6ZT0xMSksCiAgICAgICAgYXhpcy50aXRsZSA9IGVsZW1lbnRfdGV4dChzaXplID0gMTIpKSArCiAgc2NhbGVfZmlsbF9kMygpIAoKdmlvbGlucGxvdF9leHAxCgoKYm94cGxvdF9leHAxIDwtIGRhdGFfZXhwMSAlPiUKICBnZ3Bsb3QoYWVzKHg9ZmFjdG9yKHN0YWdlKSwgeT0gdHJ1dGhfZXN0aW1hdGUsIGZpbGw9IGdyb3VwKSkgKwogIGdlb21fYm94cGxvdCgpKwogICNnZW9tX3BvaW50KHBvc2l0aW9uID0gcG9zaXRpb25fZG9kZ2Uod2lkdGg9MC43NSksIGFscGhhPSAuNSkrCiAgbGFicyAoeD0gJycsIHk9ICJUcnV0aCBMaWtlbGloaG9kIEVzdGltYXRlIikgKyAKICB0aGVtZV9idygpICsgCiAgdGhlbWUobGVnZW5kLnBvc2l0aW9uID0gImJvdHRvbSIsCiAgICAgICAgYXhpcy50ZXh0PWVsZW1lbnRfdGV4dChzaXplPTExKSwKICAgICAgICBheGlzLnRpdGxlID0gZWxlbWVudF90ZXh0KHNpemUgPSAxMikpICsKICBzY2FsZV9maWxsX3NpbXBzb25zKCkgCgpib3hwbG90X2V4cDEKCgpib3hwbG90X2ZhY2V0X2V4cDEgPC0gZGF0YV9leHAxICU+JQogIGdncGxvdChhZXMoeD1mYWN0b3Ioc3RhZ2UpLCB5PSB0cnV0aF9lc3RpbWF0ZSwgZmlsbD0gZ3JvdXApKSArCiAgZ2VvbV9ib3hwbG90KCkrCiAgbGFicyAoeD0gJycsIHk9ICJUcnV0aCBMaWtlbGloaG9kIEVzdGltYXRlIikgKyAKICB0aGVtZV9idygpICsgCiAgdGhlbWUobGVnZW5kLnBvc2l0aW9uID0gImJvdHRvbSIsCiAgICAgICAgYXhpcy50ZXh0PWVsZW1lbnRfdGV4dChzaXplPTExKSwKICAgICAgICBheGlzLnRpdGxlID0gZWxlbWVudF90ZXh0KHNpemUgPSAxMiksCiAgICAgICAgYXhpcy50ZXh0LnggPSBlbGVtZW50X3RleHQoYW5nbGUgPSA5MCwgdmp1c3QgPSAwLjUsIGhqdXN0PTEpKSArCiAgZmFjZXRfd3JhcCh+Z3JvdXApKwogIHNjYWxlX2NvbG9yX3NpbXBzb25zKCkgCgpib3hwbG90X2ZhY2V0X2V4cDEKCgpgYGAKCkhvdyB0byBjb21iaW5lIG11bHRpcGxlIHBsb3RzPyBXZSBjYW4gdXNlIHRoZSBgcGF0Y2h3b3JrYCBwYWNrYWdlLiBBIG5pY2UgdHV0b3JpYWwgb24gdXNpbmcgdGhpcyBwYWNrYWdlIGNhbiBiZSBmb3VuZCBbaGVyZV0oaHR0cHM6Ly9wYXRjaHdvcmsuZGF0YS1pbWFnaW5pc3QuY29tL2FydGljbGVzL3BhdGNod29yay5odG1sKQpgYGB7ciBkcGk9IDMwMCwgZmlnLmhlaWdodD03LCBmaWcud2lkdGg9OX0KCmNvbWJpbmVkX3Bsb3RfZXhwMSA8LSAoYmFycGxvdF9mYWNldF9leHAxK2xpbmVwbG90X2V4cDEpIC8gKHZpb2xpbnBsb3RfZXhwMStib3hwbG90X2V4cDEpCmNvbWJpbmVkX3Bsb3RfZXhwMQpgYGAKCkhvdyB0byBzYXZlIGEgcGxvdD8KYGBge3JtZXNzYWdlPUZ9Cmdnc2F2ZShjb21iaW5lZF9wbG90X2V4cDEsIGZpbGVuYW1lID0gaGVyZSgib3V0cHV0cyIsImNvbWJpbmVkX3Bsb3RfZXhwMS5wbmciKSwgZHBpPTMwMCkKYGBgCgojIERhdGEgQW5hbHlzaXMKCgojIyB0LXRlc3QKCklzIHRoZXJlIGEgZGlmZmVyZW5jZSBiZXR3ZWVuIGdyb3VwcyBhdCB0aGUgZmlyc3Qgc3RhZ2U/IElkZWFsbHksIHdlIHdhbnQgcGFydGljaXBhbnRzJyByYXRpbmdzIGF0IHRoZSBmaXJzdCBzdGFnZSBiZSBzaW1pbGFyIGZvciBib3RoIGdyb3VwcyBiZWNhdXNlIHdlIGhhdmUgbm90IGRvbmUgYW55IG1hbmlwdWxhdGlvbnMuIFByZXZpb3VzIGdyYXBocyBzaG93ZWQgdXMgdGhhdCByYXRpbmdzIG9mIHNpbXBsZSBhbmQgY29tcGxleCBncm91cCBhdCB0aGlzIHN0YWdlIGFyZSBwcmV0dHkgY2xvc2UuIExldCdzIHRlc3QgdGhhdCB1c2luZyBhbiAqKmluZGVwZW5kZW50IHQtdGVzdCoqIChiZWNhdXNlIHdlIGhhdmUgMiBpbmRlcGVuZGVudCBncm91cHMpOgoKYGBge3J9CiMgSXMgdGhlcmUgYSBkaWZmZXJlbmNlIGJldHdlZW4gZ3JvdXBzIGF0IHRoZSBmaXJzdCBzdGFnZT8KZGF0YV9leHAxICU+JSAKICBncm91cF9ieShncm91cCkgJT4lIAogIGZpbHRlcihzdGFnZT09J3N0YWdlMScpICU+JSAKICB1bmdyb3VwICgpICU+JQogIHQudGVzdCh0cnV0aF9lc3RpbWF0ZX5ncm91cCwgZGF0YSA9IC4sIHBhaXJlZD1GQUxTRSkKYGBgCgpOb3csIHdlIHdvbmRlciBpZiBvcHBvc2luZyBhcmd1bWVudHMgd2VyZSBlZmZlY3RpdmUgYXQgYWxsLCByZWdhcmRsZXNzIG9mIHBhcnRpY2lwYW50cycgZ3JvdXAuIFNvLCB3ZSB3b3VsZCBsaWtlIHRvIHRlc3QgaWYgcmF0aW5ncyBhdCB0aGUgZmluYWwgc3RhZ2UgYXJlIGxvd2VyIHRoYW4gcmF0aW5ncyBhdCB0aGUgc3RhZ2UgND8gU2luY2UgYSBwYWlyIG9mIHNjb3JlIGF0IHN0YWdlIDQgYW5kIHN0YWdlIDcgaXMgY29taW5nIGZyb20gYSBzYW1lIHBlcnNvbiwgd2UgdXNlICoqcGFpcmVkIHQtdGVzdCoqLgoKYGBge3J9CiMgSXMgdGhlcmUgYSBkaWZmZXJlbmNlIGJldHdlZW4gcmF0aW5ncyBvZiBzdGFnZTQgYW5kIHN0YWdlNz8KZGF0YV9leHAxICU+JSAKICBmaWx0ZXIoc3RhZ2U9PSdzdGFnZTQnIHwgc3RhZ2U9PSdzdGFnZTcnKSAlPiUgCiAgdW5ncm91cCAoKSAlPiUKICB0LnRlc3QodHJ1dGhfZXN0aW1hdGV+c3RhZ2UsIGRhdGEgPSAuLCBwYWlyZWQ9VFJVRSkKYGBgCgoKIyMjIEV4ZXJjaXNlCgpKb2huIGV0IGFsLiAoMjAxOSkgaW52ZXN0aWdhdGVkIHRoZSBjb25zZXF1ZW5jZXMgb2YgYmFja2luZyBkb3duIChjaGFuZ2luZyBvbmUncyBtaW5kIGluIGxpZ2h0cyBvZiBldmlkZW5jZSlhbmQgaG93IG90aGVyIHBlb3BsZSB2aWV3IHNvbWVvbmUgd2hvIGNoYW5nZSB0aGVpciBtaW5kLiBJbiB0aGVpciBzZWNvbmQgZXhwZXJpbWVudHMsIHRoZXkgcHJlc2VudGVkIHBhcnRpY2lwYW50cyBlaXRoZXIgd2l0aCBhIHBlcnNvbiB3aG8gY2hhbmdlcyB0aGVpciBtaW5kIG9yIGEgcGVyc29uIHdobyByZWZ1c2VzIHRvIGJhY2sgZG93bi4gVGhlbiwgdGhleSBhc2tlZCBwYXJ0aWNpcGFudHMgdG8gcmF0ZSBob3cgaW50ZWxsaWdlbnQgYW5kIGNvbmZpZGVudCB0aGUgcGVyc29uIGlzIChTZWUgdGhlIG9yaWdpbmFsIHN0dWR5IFtoZXJlXShodHRwczovL3d3dy5oYnMuZWR1L2ZhY3VsdHkvUHVibGljYXRpb24lMjBGaWxlcy9Kb2huJTIwZXQlMjBhbCUyMC0lMjBzZWxmLXByZXNlbnRhdGlvbmFsJTIwY29uc2VxdWVuY2VzX2I4NWIyYzQzLWE1YjUtNDc0Yy05ZTJjLWU5ODUzYjEwNzI3ZS5wZGYpKS4gVGhleSByZXBvcnRlZCB0aGF0OiAKCj4gIlJlbGF0aXZlIHRvIHRoZSBlbnRyZXByZW5ldXIgd2hvIGRpZCBub3QgYmFjayBkb3duLCBwYXJ0aWNpcGFudHMganVkZ2VkIHRoZSBlbnRyZXByZW5ldXIgd2hvIGJhY2tlZCBkb3duIGFzIG1vcmUgaW50ZWxsaWdlbnQgKE1fYmFja2VkX2Rvd249NS4xMyBvdXQgb2YgNywgU0Q9MS4wOTsgTV9kaWRfbm90X2JhY2tfZG93bj0zLjk3LCBTRD0xLjU0OyB0KDI3MS4xMik94oiSNy41OSwgcCA8IC4wMDEpIGJ1dCBsZXNzIGNvbmZpZGVudCAoTV9iYWNrZWRfZG93bj00LjUwIG91dCBvZiA3LCBTRD0xLjM2OyBNX2RpZF9ub3RfYmFja19kb3duPTUuNjUsIFNEPTEuMTA7IHQoMjkxLjAxKT04LjA4LCBwIDwgLjAwMSkuIi4KCk9wZW4gdGhlIGBqb2huX2JhY2tkb3duX2V4cDIuY3N2YCBmaWxlIGFuZCB0cnkgdG8gcmVwcm9kdWNlIHRoZWlyIHJlc3VsdHMuIFJ1biB0d28gc2VwYXJhdGUgaW5kZXBlbmRlbnQgdC10ZXN0LCBvbmUgd2l0aCBgaW50ZWxsaWdlbnRgIGFzIHRoZSBkZXBlbmRlbnQgdmFyaWFibGUgYW5kIG9uZSB3aXRoIGBjb25maWRlbnRgIGFzIHRoZSBkZXBlbmRlbnQgdmFyaWFibGUuIEZvciBib3RoIHQtdGVzdCwgdXNlIGBiYWNrX2Rvd25gIGFzIHRoZSBiZXR3ZWVuLXN1YmplY3QgaW5kZXBlbmRlbnQgdmFyaWFibGUuCgpgYGB7ciBtZXNzYWdlPUYsIHdhcm5pbmc9Rn0Kam9obl9kYXRhIDwtIHJlYWRfY3N2KGhlcmUoImNsZWFuZWRfZGF0YSIsImpvaG5fYmFja2Rvd25fZXhwMi5jc3YiKSkKCgp0LnRlc3QoaW50ZWxsaWdlbnR+YmFja19kb3duLCBkYXRhID0gam9obl9kYXRhLCBwYWlyZWQ9RkFMU0UpCnQudGVzdChjb25maWRlbnR+YmFja19kb3duLCBkYXRhID0gam9obl9kYXRhLCBwYWlyZWQ9RkFMU0UpCmBgYAoKCiMjIEFuYWx5c2lzIG9mIFZhcmlhbmNlIChBTk9WQSkKCk5vdywgbGV0J3MgYW5zd2VyIG91ciBtYWluIHF1ZXN0aW9uOiBEbyBwYXJ0aWNpcGFudHMgaW4gdGhlIHNpbXBsZSBncm91cCBzaG93IGhpZ2hlciByYXRpbmdzIGZvciBzdXBwb3J0aXZlIGFyZ3VtZW50cyAoc3RhZ2UgMiB0byA0KSBhbmQgbG93ZXIgcmF0aW5ncyBmb3Igb3Bwb3NpbmcgYXJndW1lbnRzIChzdGFnZSA1IHRvIDcpLCBjb21wYXJlZCB0byBwYXJ0aWNpcGFudHMgaW4gdGhlIGNvbXBsZXggZ3JvdXA/IElmIHRoaXMgaXMgdGhlIGNhc2UuIHdlIGV4cGVjdCBhbiBpbnRlcmFjdGlvbiBpbiB0aGUgdHJhZGl0aW9uYWwgKipBbmFseXNpcyBvZiBWYXJpYW5jZSAoQU9OVkEpKiogdGVzdC4KCmBgYHtyIG1lc3NhZ2U9Riwgd2FybmluZz1GfQphb3ZfbTEgPC0gYW92X2NhciAodHJ1dGhfZXN0aW1hdGUgfiBncm91cCpzdGFnZSArCiAgICAgICAgICAgICAgICAgICAgIEVycm9yKHN1YmplY3Qvc3RhZ2UpLCBkYXRhID0gZGF0YV9leHAxKQpgYGAKCmBgYHtyIGVjaG89Rn0Ka25pdHI6OmthYmxlKG5pY2UoYW92X20xKSkgJT4lCiAga2FibGVfc3R5bGluZyhib290c3RyYXBfb3B0aW9ucyA9IGMoInN0cmlwZWQiLCAiYm9yZGVyZWQiLCAiY29uZGVuc2VkIiksIGZpeGVkX3RoZWFkID0gVCwgZnVsbF93aWR0aCA9IFQpCmBgYAoKQXMgeW91IGNhbiBzZWUsIHdlIGZvdW5kIGEgc2lnbmlmaWNhbnQgbWFpbiBlZmZlY3Qgb2Ygc3RhZ2UgYW5kIGEgc2lnbmlmaWNhbnQgZ3JvdXAgYnkgc3RhZ2UgaW50ZXJhY3Rpb24uIFdlIGNhbiB1c2UgdGhlIGBlbW1lYW5zYCBwYWNrYWdlIHRvIGRvIHBvc3QtaG9jIHRlc3RzLgoKYGBge3Igd2FybmluZz1GLCBtZXNzYWdlPUZ9CiMgbWFpbiBlZmZlY3Qgb2Ygc3RhZ2UKZW1tZWFucyhhb3ZfbTEsICdzdGFnZScpCnBhaXJzKGVtbWVhbnMoYW92X20xLCAnc3RhZ2UnKSwgYWRqdXN0PSAnaG9sbScpCmBgYAoKCmBgYHtyIHdhcm5pbmc9RiwgbWVzc2FnZT1GfQojIGdyb3VwIGJ5IHN0YWdlIGludGVyYWN0aW9uCmVtbWVhbnMoYW92X20xLCAiZ3JvdXAiLCBieT0gInN0YWdlIikKdXBkYXRlKHBhaXJzKGVtbWVhbnMoYW92X20xLCAiZ3JvdXAiLCBieT0gInN0YWdlIikpLCBieSA9IE5VTEwsIGFkanVzdCA9ICJob2xtIikgCmBgYAoKWW91IGNhbiB1c2UgdGhlIGBhZmV4X3Bsb3RgIGZ1bmN0aW9uIGZyb20gYWZleCB0byBjcmVhdGUgYmVhdXRpZnVsIHBsb3RzLiBUaG9zZSBwbG90cyBpbnRlcmFjdHMgbmljZWx5IHdpdGggZ2dwbG90OgpgYGB7ciBtZXNzYWdlPUYsIHdhcm5pbmc9RiwgZHBpPSAzMDB9CmFmZXhfcGxvdChhb3ZfbTEsIHggPSAic3RhZ2UiLCB0cmFjZSA9ICJncm91cCIsIGVycm9yPSdiZXR3ZWVuJywKICAgICAgICAgIGxpbmVfYXJnID0gbGlzdChzaXplPTEpLAogICAgICAgICAgcG9pbnRfYXJnID0gbGlzdChzaXplPTMuNSksCiAgICAgICAgICBkYXRhX2FyZyA9IGxpc3Qoc2l6ZT0gMSwgY29sb3I9ICdncmV5Jywgd2lkdGg9LjQpLAogICAgICAgICAgZGF0YV9nZW9tID0gZ2VvbV9ib3hwbG90LAogICAgICAgICAgbWFwcGluZyA9IGMoImxpbmV0eXBlIiwgInNoYXBlIiwgImZpbGwiKSwKICAgICAgICAgIGxlZ2VuZF90aXRsZSA9ICJHcm91cCIpICsKICBsYWJzKHkgPSAiVHJ1dGggTGlrZWxpaGhvZCBFc3RpbWF0ZSIsIHggPSAiIikgKwogIHRoZW1lX2J3KCkrICMgcmVtb3ZlIHRoZSBncmV5IGJhY2tncm91bmQgYW5kIGdyaWQKICB0aGVtZShheGlzLnRleHQ9ZWxlbWVudF90ZXh0KHNpemU9MTMpLAogICAgICAgIGF4aXMudGl0bGUgPSBlbGVtZW50X3RleHQoc2l6ZSA9IDEzKSwKICAgICAgICBsZWdlbmQudGV4dD1lbGVtZW50X3RleHQoc2l6ZT0xMyksCiAgICAgICAgbGVnZW5kLnRpdGxlPWVsZW1lbnRfdGV4dChzaXplPTEzKSwKICAgICAgICBsZWdlbmQucG9zaXRpb249J2JvdHRvbScsCiAgICAgICAgbGVnZW5kLmtleS5zaXplID0gdW5pdCgxLCAiY20iKSwKICAgICAgICBsZWdlbmQuYmFja2dyb3VuZCA9IGVsZW1lbnRfcmVjdChjb2xvdXIgPSAnYmxhY2snLCBmaWxsID0gJ3doaXRlJywgbGluZXR5cGU9J3NvbGlkJykpKwogIHNjYWxlX2NvbG9yX3NpbXBzb25zKCkgKwogIHNjYWxlX2ZpbGxfc2ltcHNvbnMoKQpgYGAKCgpJZiB5b3UgYXJlIGludGVyZXN0ZWQgaW4gdGhpcyB0b3BpYywgY2hlY2sgb3V0IHRoaXMgbmljZSB0dXRvcmlhbCBhYm91dCBbdXNpbmcgYWZleCB0byBydW4gQU5PVkFdKGh0dHBzOi8vY3Jhbi5yLXByb2plY3Qub3JnL3dlYi9wYWNrYWdlcy9hZmV4L3ZpZ25ldHRlcy9hZmV4X2Fub3ZhX2V4YW1wbGUuaHRtbCksIGFuZCBhbHNvIHRoaXMgaW50ZXJlc3RpbmcgdHV0b3JpYWwgb24gdGhlIFtlbW1lYW5zIHBhY2thZ2VdKGh0dHBzOi8vYW9zbWl0aC5yYmluZC5pby8yMDE5LzAzLzI1L2dldHRpbmctc3RhcnRlZC13aXRoLWVtbWVhbnMvKS4KCiMjIyBFeGVyY2lzZQoKUm90ZWxsbyBldCBhbC4gKDIwMTgpIGludmVzdGlnYXRlZCB0aGUgYXNzb2NpYXRpb24gYmV0d2VlbiB0aGUgcmFjZSAoV2hpdGUgdnMuIEJsYWNrIGZhY2VzKSBhbmQgdGhlIGd1bi10b29sIGp1ZGdtZW50cy4gSW4gdGhlaXIgZmlyc3QgZXhwZXJpbWVudHMsIHRoZXkgcHJlc2VudGVkIHBhcnRpY2lwYW50cyB3aXRoIDE2IFdoaXRlIG1hbGUgZmFjZXMgYW5kIDE2IEJsYWNrIG1hbGUgZmFjZXMsIGFuZCBmb2xsb3dpbmcgdGhhdCA4IGltYWdlcyBvZiBndW5zIGFuZCA4IGltYWdlcyBvZiB0b29scy4gVGhleSBhc2tlZCBwYXJ0aWNpcGFudHMgdG8ganVkZ2UgaWYgdGhlIG9iamVjdCBpcyBhIHRvb2wgb3IgYSBndW4gYnkgcHJlc3Npbmcga2V5Ym9hcmQgYnV0dG9ucy4gVGhlbiwgdGhleSByYW4gYW4gQU5PVkEgdG8gc2VlIGlmIHBhcnRpY2lwYW50cycgZ3VuIHJlc3BvbnNlcyBhcmUgaGlnaGVyIGZvciBhbnkgb2YgdGhlIHJhY2VzLiBTbywgdGhleSBpbmNsdWRlZCBwcmltZSByYWNlIChCbGFjaywgV2hpdGUpIGFuZCB0YXJnZXQgaWRlbnRpdHkgKGd1biwgdG9vbCkgYXMgaW5kZXBlbmRlbnQgdmFyaWFibGVzIGFuZCBwYXJ0aWNpcGFudHMnIGd1biByZXNwb25zZXMgYXMgZGVwZW5kZW50IHZhcmlhYmxlIGludG8gdGhlaXIgbGluZWFyIG1vZGVsIChTZWUgdGhlIG9yaWdpbmFsIHN0dWR5IFtoZXJlXShodHRwczovL3BzeWFyeGl2LmNvbS9hN2s5NikpLiBUaGV5IGZvdW5kIHRoYXQ6IAoKPiAiUGFydGljaXBhbnRzIG1hZGUgbW9yZSBndW4gcmVzcG9uc2VzIHRvIGd1bnMgdGhhbiB0byB0b29scywgRigxLDQ1KSA9IDUzMjQzLCBwIDwgMC4wMDAxLCDOtzJnID0gMC45OTguIEhvd2V2ZXIsIHRoZSByYWNlIG9mIHRoZSBwcmltZSBmYWNlIGRpZCBub3QgbWF0dGVyLCBGKDEsNDUpID0gMC4yODcsIHAgPiAwLjU5LCDOtzJnID0gMC4wMDEsIG5vciB3YXMgdGhlcmUgYW4gaW50ZXJhY3Rpb24gb2YgcHJpbWUgcmFjZSB3aXRoIHRhcmdldCBvYmplY3QsIEYoMSw0NSkgPSAwLjAyMiwgcCA+IDAuODgsIM63MmcgPSAwLjAwMCkiLgoKT3BlbiB0aGUgYHJvdGVsbG9fc2hvb3Rlcl9leHAxLmNzdmAgZmlsZSBhbmQgdHJ5IHRvIHJlcHJvZHVjZSB0aGVpciByZXN1bHRzLiBSdW4gYW4gQU5PVkEgKHR5cGUgSUlJKSB3aXRoIGByZXNwYCBhcyB0aGUgZGVwZW5kZW50IHZhcmlhYmxlIGFuZCB0YXJnZXQsIHByaW1lLCBhbmQgdGhlaXIgaW50ZXJhY3Rpb24gYXMgaW5kZXBlbmRlbnQgdmFyaWFibGVzLgoKCmBgYHtyIG1lc3NhZ2U9Riwgd2FybmluZz1GfQojIGxvYWQgdGhlIGdlbmVyYWwgZGF0YSBmaWxlCnJvdGVsbG9fZGF0YSA8LSByZWFkX2NzdihoZXJlKCJjbGVhbmVkX2RhdGEiLCJyb3RlbGxvX3Nob290ZXJfZXhwMS5jc3YiKSkKCiMgQU5PVkEKcm90ZWxsb19hb3YgPC0gYW92X2NhciAocmVzcCB+IHRhcmdldCpwcmltZSArCiAgICAgICAgICAgRXJyb3Ioc3ViamVjdC90YXJnZXQqcHJpbWUpLCBkYXRhID0gcm90ZWxsb19kYXRhKQpgYGAKCmBgYHtyIGVjaG89Rn0Ka25pdHI6OmthYmxlKG5pY2Uocm90ZWxsb19hb3YpKSAlPiUKICBrYWJsZV9zdHlsaW5nKGJvb3RzdHJhcF9vcHRpb25zID0gYygic3RyaXBlZCIsICJib3JkZXJlZCIsICJjb25kZW5zZWQiKSwgZml4ZWRfdGhlYWQgPSBULCBmdWxsX3dpZHRoID0gVCkKYGBgCgoKCiMjIENvcnJlbGF0aW9uCgpOb3csIGxldCdzIGFuc3dlciB0byBhbm90aGVyIHF1ZXN0aW9uIG9mIHRoaXMgc3R1ZHk6IGRvZXMgcGVyc3Vhc2lvbiBhbmQgZGlzc3Vhc2lvbiBpcyByZWxhdGVkIHRvIG9wZW4tbWluZGVkbmVzcywgY29nbml0aXZlIGFiaWxpdHksIHJlYXNvbmluZyBhYmlsaXRpZXMsIGFuZCBjb2duaXRpdmUgc3R5bGU/IFRvIGFuc3dlciB0aGlzIHF1ZXN0aW9uLCB3ZSBuZWVkIHRvIGNyZWF0ZSB0d28gaW5kZXhlcyAoc2NvcmVzKSBvbmUgZm9yIHBlcnN1YXNpb24gYW5kIG9uZSBmb3IgZGlzc3Vhc2lvbi4gVGhlbiB3ZSBjYW4gZG8gYSBjb3JyZWxhdGlvbiB0ZXN0OgoKYGBge3IgbWVzc2FnZT1GLCBldmFsPUYsIGZpZy5hbGlnbj0nY2VudGVyJywgZHBpPTMwMH0KY29yX2RhdGFfZXhwMSA8LSBkYXRhX2V4cDEgJT4lIAogIHBpdm90X3dpZGVyKG5hbWVzX2Zyb20gPSBzdGFnZSwgdmFsdWVzX2Zyb20gPSB0cnV0aF9lc3RpbWF0ZSkgJT4lCiAgZ3JvdXBfYnkoc3ViamVjdCkgJT4lCiAgbXV0YXRlKHBlcnN1YXNpb25faW5kZXg9IHN0YWdlMisgc3RhZ2UzKyBzdGFnZTQgLSBzdGFnZTEsCiAgICAgICAgIGRpc3N1YXNpb25faW5kZXg9ICgxMDEtc3RhZ2U1KSArICgxMDEtc3RhZ2U2KSArICgxMDEtc3RhZ2U3KSAtICgxMDEtc3RhZ2U0KSkgJT4lCiAgdW5ncm91cCgpJT4lCiAgZHBseXI6OnNlbGVjdChwZXJzdWFzaW9uX2luZGV4LGRpc3N1YXNpb25faW5kZXgsYW90ZV90b3RhbCxudW1lcmFjeV90b3RhbCxjcnRfdG90YWwsbWluZHdhcmVfdG90YWwpCgojLS0tLS0tLS0tLSBCYXNlIFI6CmNvcihjb3JfZGF0YV9leHAxLCBtZXRob2QgPSAicGVhcnNvbiIsICB1c2UgPSAiY29tcGxldGUub2JzIikKCiMtLS0tLS0tLS0tIFBzeWNoIGxpYnJhcnk6CmNvcl9kYXRhX2V4cDEgJT4lIAogIHBzeWNoOjpwYWlycy5wYW5lbHMobWV0aG9kID0gInBlYXJzb24iLCBoaXN0LmNvbCA9ICIjMDBBRkJCIiwgZGVuc2l0eSA9IFQsIGVsbGlwc2VzID0gRiwgc3RhcnMgPSBUKQoKIy0tLS0tLS0tLS0gQ29ycmVsYXRpb24gbGlicmFyeToKY29ycmVsYXRpb246OmNvcnJlbGF0aW9uKGNvcl9kYXRhX2V4cDEpICU+JSBzdW1tYXJ5KCkKCiMtLS0tLS0tLS0tIGFwYVRhYmxlcyBsaWJyYXJ5Ogpjb3JfZGF0YV9leHAxICU+JSAKICBhcGFUYWJsZXM6OmFwYS5jb3IudGFibGUoZmlsZW5hbWU9Ii4vb3V0cHV0cy9Db3JNYXRyaXguZG9jIiwgc2hvdy5jb25mLmludGVydmFsPVQpCmBgYAoKYGBge3IgbWVzc2FnZT1GLCBlY2hvPUYsIGZpZy5hbGlnbj0nY2VudGVyJywgZHBpPTMwMH0KY29yX2RhdGFfZXhwMSA8LSBkYXRhX2V4cDEgJT4lIAogIHBpdm90X3dpZGVyKG5hbWVzX2Zyb20gPSBzdGFnZSwgdmFsdWVzX2Zyb20gPSB0cnV0aF9lc3RpbWF0ZSkgJT4lCiAgZ3JvdXBfYnkoc3ViamVjdCkgJT4lCiAgbXV0YXRlKHBlcnN1YXNpb25faW5kZXg9IHN0YWdlMisgc3RhZ2UzKyBzdGFnZTQgLSBzdGFnZTEsCiAgICAgICAgIGRpc3N1YXNpb25faW5kZXg9ICgxMDEtc3RhZ2U1KSArICgxMDEtc3RhZ2U2KSArICgxMDEtc3RhZ2U3KSAtICgxMDEtc3RhZ2U0KSkgJT4lCiAgdW5ncm91cCgpJT4lCiAgZHBseXI6OnNlbGVjdChwZXJzdWFzaW9uX2luZGV4LGRpc3N1YXNpb25faW5kZXgsYW90ZV90b3RhbCxudW1lcmFjeV90b3RhbCxjcnRfdG90YWwsbWluZHdhcmVfdG90YWwpCgojLS0tLS0tLS0tLSBCYXNlIFI6CmNvcihjb3JfZGF0YV9leHAxLCBtZXRob2QgPSAicGVhcnNvbiIsICB1c2UgPSAiY29tcGxldGUub2JzIiklPiUKICBrbml0cjo6a2FibGUoZGlnaXRzID0gMikgJT4lCiAga2FibGVfc3R5bGluZyhib290c3RyYXBfb3B0aW9ucyA9IGMoInN0cmlwZWQiLCAiYm9yZGVyZWQiLCAiY29uZGVuc2VkIiksIGZpeGVkX3RoZWFkID0gVCwgZnVsbF93aWR0aCA9IEYpJT4lCiAgc2Nyb2xsX2JveCh3aWR0aCA9ICI3ODBweCIpCgojLS0tLS0tLS0tLSBQc3ljaCBsaWJyYXJ5Ogpjb3JfZGF0YV9leHAxICU+JSAKICBwc3ljaDo6cGFpcnMucGFuZWxzKG1ldGhvZCA9ICJwZWFyc29uIiwgaGlzdC5jb2wgPSAiIzAwQUZCQiIsIGRlbnNpdHkgPSBULCBlbGxpcHNlcyA9IEYsIHN0YXJzID0gVCkKCiMtLS0tLS0tLS0tIENvcnJlbGF0aW9uIGxpYnJhcnk6CmNvcnJlbGF0aW9uOjpjb3JyZWxhdGlvbihjb3JfZGF0YV9leHAxKSAlPiUgc3VtbWFyeSgpJT4lCiAga25pdHI6OmthYmxlKGRpZ2l0cyA9IDIpICU+JQogIGthYmxlX3N0eWxpbmcoYm9vdHN0cmFwX29wdGlvbnMgPSBjKCJzdHJpcGVkIiwgImJvcmRlcmVkIiwgImNvbmRlbnNlZCIpLCBmaXhlZF90aGVhZCA9IFQsIGZ1bGxfd2lkdGggPSBGKSU+JQogIHNjcm9sbF9ib3god2lkdGggPSAiNzgwcHgiKQoKYGBgCgoKCiMjIyBFeGVyY2lzZQoKUGVubnljb29rIGV0IGFsLiAoMjAyMCkgaW52ZXN0aWdhdGVkIHRoZSByZWxhdGlvbnNoaXAgYmV0d2VlbiBhY3RpdmVseSBvcGVuLW1pbmRlZCB0aGlua2luZyBzdHlsZSBhYm91dCBldmlkZW5jZSAoQU9ULUUpIGFuZCBkaWZmZXJlbnQgcG9saXRpY2FsLCBzY2llbnRpZmljLCBhbmQgcmVsaWdpb3VzIGJlbGllZnMgKHNlZSB0aGUgb3JpZ2luYWwgcGFwZXIgW2hlcmVdKGh0dHBzOi8vcHN5YXJ4aXYuY29tL2E3azk2KSkuIEluIHRoZWlyIGZpcnN0IGV4cGVyaW1lbnQsIHRoZXkgY2FsY3VsYXRlZCB0aGUgY29ycmVsYXRpb24gb2YgQU9URSBhbmQgc2NpZW50aWZpYyBiZWxpZWZzIGl0ZW1zIChnbG9iYWwgd2FybWluZywgZXZvbHV0aW9uLCBldGMuKSBhbmQgdGhleSBmb3VuZCB0aGUgZm9sbG93aW5nIHJlc3VsdHM6CgpgYGB7ciBlY2hvPUZBTFNFLCBvdXQud2lkdGg9IjcwMHB4Iiwgb3V0LmhlaWdodD0iMzUwcHgiLCBmaWcuY2FwPSAiYWRhcHRlZCBmcm9tIFtQZW5ueWNvb2sgZXQgYWwuICgyMDIwKV0oaHR0cHM6Ly9wc3lhcnhpdi5jb20vYTdrOTYpIn0Ka25pdHI6OmluY2x1ZGVfZ3JhcGhpY3MoaGVyZSgnaW5wdXRzJywncGVubnljb29rX2NvcnIucG5nJykpCmBgYAoKT3BlbiB0aGUgYHBlbm55Y29va19hb3RlX2V4cDEuY3N2YCBmaWxlIGFuZCB0cnkgdG8gcmVwcm9kdWNlIHRoZWlyIHJlc3VsdHMgYnkgY3JlYXRpbmcgdGhlIHNhbWUgY29ycmVsYXRpb24gbWF0cml4LgoKYGBge3IgbWVzc2FnZT1GLCBldmFsPUZ9CnBlbm55Y29va19kYXRhIDwtIHJlYWRfY3N2KGhlcmUoImNsZWFuZWRfZGF0YSIsInBlbm55Y29va19hb3RlX2V4cDEuY3N2IikpIAoKCiMtLS0tLS0tLS0tIEJhc2UgUjoKY29yKHBlbm55Y29va19kYXRhLCBtZXRob2QgPSAicGVhcnNvbiIsICB1c2UgPSAiY29tcGxldGUub2JzIikKCiMtLS0tLS0tLS0tIFBzeWNoIGxpYnJhcnk6CnBlbm55Y29va19kYXRhICU+JSAKICBwc3ljaDo6cGFpcnMucGFuZWxzKG1ldGhvZCA9ICJwZWFyc29uIiwgaGlzdC5jb2wgPSAiIzAwQUZCQiIsIGRlbnNpdHkgPSBULCBlbGxpcHNlcyA9IEYsIHN0YXJzID0gVCkKCiMtLS0tLS0tLS0tIENvcnJlbGF0aW9uIGxpYnJhcnk6CmNvcnJlbGF0aW9uOjpjb3JyZWxhdGlvbihwZW5ueWNvb2tfZGF0YSkgJT4lIHN1bW1hcnkoKQoKIy0tLS0tLS0tLS0gYXBhVGFibGVzIGxpYnJhcnk6CnBlbm55Y29va19kYXRhICU+JSAKICBhcGFUYWJsZXM6OmFwYS5jb3IudGFibGUoZmlsZW5hbWU9Ii4vb3V0cHV0cy9Db3JNYXRyaXguZG9jIiwgc2hvdy5jb25mLmludGVydmFsPVQpCmBgYAoKCmBgYHtyIG1lc3NhZ2U9RiwgZXZhbD1ULCBlY2hvPUYsIGZpZy5hbGlnbj0nY2VudGVyJywgZHBpPTMwMH0KcGVubnljb29rX2RhdGEgPC0gcmVhZF9jc3YoaGVyZSgiY2xlYW5lZF9kYXRhIiwicGVubnljb29rX2FvdGVfZXhwMS5jc3YiKSkgJT4lCiAgY2xlYW5fbmFtZXMoKQoKY29ycmVsYXRpb246OmNvcnJlbGF0aW9uKHBlbm55Y29va19kYXRhKSAlPiUgc3VtbWFyeSgpICU+JQogIGtuaXRyOjprYWJsZShkaWdpdHMgPSAyKSAlPiUKICBrYWJsZV9zdHlsaW5nKGJvb3RzdHJhcF9vcHRpb25zID0gYygic3RyaXBlZCIsICJib3JkZXJlZCIsICJjb25kZW5zZWQiKSwgZml4ZWRfdGhlYWQgPSBULCBmdWxsX3dpZHRoID0gRiklPiUKICBzY3JvbGxfYm94KHdpZHRoID0gIjc4MHB4IikKCmBgYAoKCiMjIExpbmVhciBSZWdyZXNzaW9uCgpJbiB0aGUgcHJldmlvdXMgc2VjdGlvbiwgd2UgZm91bmQgdGhhdCBvcGVuLW1pbmRlZG5lc3MgKEFPVC1FKSBpcyBjb3JyZWxhdGVkIHdpdGggcGVyc3Vhc2lvbi4gTm93LCBvbmUgbWF5IGFzayBpZiBvcGVuLW1pbmRlZG5lc3MgY2FuIHByZWRpY3QgcGVyc3Vhc2lvbiBhZnRlciBjb250cm9sbGluZyBmb3IgcmVhc29uaW5nIGFuZCBjb250cm9sbGluZyBhYmlsaXRpZXM/IFRvIGFuc3dlciB0aGF0LCB3ZSBjYW4gcnVuIGEgbXVsdGlwbGUgcmVncmVzc2lvbiBhbmFseXNpczoKYGBge3J9CmV4cDFfcmVnPWxtKHBlcnN1YXNpb25faW5kZXggfiBhb3RlX3RvdGFsKyBudW1lcmFjeV90b3RhbCsgY3J0X3RvdGFsKyBtaW5kd2FyZV90b3RhbCwKICAgICAgICAgICAgICAgICAgZGF0YT1jb3JfZGF0YV9leHAxKQpgYGAKCmBgYHtyIG1lc3NhZ2U9RiwgZXZhbD1ULCBlY2hvPUYsIGZpZy5hbGlnbj0nY2VudGVyJywgZHBpPTMwMH0KYnJvb206OnRpZHkoZXhwMV9yZWcpJT4lCiAga25pdHI6OmthYmxlKGRpZ2l0cyA9IDIpICU+JQogIGthYmxlX3N0eWxpbmcoYm9vdHN0cmFwX29wdGlvbnMgPSBjKCJzdHJpcGVkIiwgImJvcmRlcmVkIiwgImNvbmRlbnNlZCIpLCBmaXhlZF90aGVhZCA9IFQsIGZ1bGxfd2lkdGggPSBUKQpgYGAKCiMjIyBFeGVyY2lzZQoKVHLDqW1vbGnDqHJlIGFuZCBEamVyaW91YXQgKDIwMjApIGV4YW1pbmVkIHRoZSByb2xlIG9mICpjb2duaXRpdmUgcmVmbGVjdGlvbiogYW5kICpiZWxpZWYgaW4gc2NpZW5jZSogaW4gY2xpbWF0ZSBjaGFuZ2Ugc2tlcHRpY2lzbS4gSW4gdGhlaXIgZmlyc3Qgc3R1ZHksIHRoZXkgcmV2ZWFsZWQgdGhhdCBjb2duaXRpdmUgcmVmbGVjdGlvbiBhbmQgYmVsaWVmIGluIHNjaWVuY2UgbmVnZXRpdmVseSBwcmVkaWN0ZWQgY2xpbWF0ZSBjaGFuZ2Ugc2tlcHRpY2lzbSBldmVuIGFmdGVyIGNvbnRyb2xsaW5nIGZvciBkZW1vZ3JhcGhpYyBhbmQgY29nbml0aXZlIGFiaWxpdHkgdmFyaWFibGVzIChzZWUgdGhlIG9yaWdpbmFsIHBhcGVyIFtoZXJlXShodHRwczovL3BzeWFyeGl2LmNvbS92cDhrNi8pKS4gCgpgYGB7ciBlY2hvPUZBTFNFLCBvdXQud2lkdGg9IjcwMHB4Iiwgb3V0LmhlaWdodD0iMzUwcHgiLCBmaWcuY2FwPSAiYWRhcHRlZCBmcm9tIFtUcsOpbW9sacOocmUgYW5kIERqZXJpb3VhdCAoMjAyMCldKGh0dHBzOi8vcHN5YXJ4aXYuY29tL3ZwOGs2LykifQprbml0cjo6aW5jbHVkZV9ncmFwaGljcyhoZXJlKCdpbnB1dHMnLCd0cmVtb2xpZXJlX3JlZy5wbmcnKSkKYGBgCgpPcGVuIHRoZSBgdHJlbW9saWVyZV9kYXRhX2V4cDEuY3N2YCBmaWxlIGFuZCB0cnkgdG8gcmVwcm9kdWNlIHRoZWlyIHJlc3VsdHMgYnkgcnVubmluZyBhIG11bHRpcGxlIGxpbmVhciByZWdyZXNzaW9uLiBFbnRlciBhZ2UsIGdlbmRlciwgZWR1Y2F0aW9uLCBiZWxpZWYgaW4gc2NpZW5jZSwgbGl0ZXJhY3ksIG51bWVyYWN5IChOdW10b3RhbCksIGFuZCBjb2duaXRpdmUgcmVmbGVjdGlvbiBhcyBwcmVkaWN0b3JzIGFuZCBlbnRlciBjbGltYXRlIGNoYW5nZSBza2VwdGljaXNtIChjbGltYXRvKSBhcyB0aGUgb3V0Y29tZSB2YXJpYWJsZS4KCmBgYHtyIG1lc3NhZ2U9Rn0KVHJlbW9saWVyZV9kYXRhIDwtIHJlYWRfY3N2KGhlcmUoImNsZWFuZWRfZGF0YSIsInRyZW1vbGllcmVfZGF0YV9leHAxLmNzdiIpKQoKVHJlbW9saWVyZV9yZWc9bG0oQ2xpbWF0byB+IEFnZSsgR2VuZGVyKyBFZHVjYXRpb24rIEJlbGllZkluU2NpZW5jZXRvdGFsKyBMaXRlcmFjeSsgTnVtdG90YWwrIENvZ25pdGl2ZVJlZmxlY3Rpb24sCiAgICAgICAgICAgICAgICAgICAgZGF0YT1UcmVtb2xpZXJlX2RhdGEpCmBgYAoKCmBgYHtyIG1lc3NhZ2U9RiwgZXZhbD1ULCBlY2hvPUYsIGZpZy5hbGlnbj0nY2VudGVyJywgZHBpPTMwMH0KYnJvb206OnRpZHkoVHJlbW9saWVyZV9yZWcpJT4lCiAga25pdHI6OmthYmxlKGRpZ2l0cyA9IDIpICU+JQogIGthYmxlX3N0eWxpbmcoYm9vdHN0cmFwX29wdGlvbnMgPSBjKCJzdHJpcGVkIiwgImJvcmRlcmVkIiwgImNvbmRlbnNlZCIpLCBmaXhlZF90aGVhZCA9IFQsIGZ1bGxfd2lkdGggPSBUKQoKZ2xhbmNlKFRyZW1vbGllcmVfcmVnKSU+JQogIGtuaXRyOjprYWJsZShkaWdpdHMgPSAyKSAlPiUKICBrYWJsZV9zdHlsaW5nKGJvb3RzdHJhcF9vcHRpb25zID0gYygic3RyaXBlZCIsICJib3JkZXJlZCIsICJjb25kZW5zZWQiKSwgZml4ZWRfdGhlYWQgPSBULCBmdWxsX3dpZHRoID0gRiklPiUKICBzY3JvbGxfYm94KHdpZHRoID0gIjc4MHB4IikKYGBgCgoKIyBSbWFya2Rvd24KClRvIGJlIGNvbXBsZXRlZC4uLgoKCiMgUmVmZXJlbmNlcwoKLSBHaGFzZW1pLCBPLiwgSGFuZGxleSwgUy4sICYgSG93YXJ0aCwgUy4gKDIwMjApLiBUaGUgQnJpZ2h0IEhvbXVuY3VsdXMgaW4gb3VyIEhlYWQ6IEluZGl2aWR1YWwgRGlmZmVyZW5jZXMgaW4gSW50dWl0aXZlIFNlbnNpdGl2aXR5IHRvIExvZ2ljYWwgVmFsaWRpdHkuCgotIEpvaG4sIEwuIEsuLCBKZW9uZywgTS4sIEdpbm8sIEYuLCAmIEh1YW5nLCBMLiAoMjAxOSkuIFRoZSBzZWxmLXByZXNlbnRhdGlvbmFsIGNvbnNlcXVlbmNlcyBvZiB1cGhvbGRpbmcgb25l4oCZcyBzdGFuY2UgaW4gc3BpdGUgb2YgdGhlIGV2aWRlbmNlLiBPcmdhbml6YXRpb25hbCBCZWhhdmlvciBhbmQgSHVtYW4gRGVjaXNpb24gUHJvY2Vzc2VzLCAxNTQsIDEtMTQuCgotIFBlbm55Y29vaywgRy4sIENoZXluZSwgSi4gQS4sIEtvZWhsZXIsIEQuIEouLCAmIEZ1Z2Vsc2FuZywgSi4gQS4gKDIwMjApLiBPbiB0aGUgYmVsaWVmIHRoYXQgYmVsaWVmcyBzaG91bGQgY2hhbmdlIGFjY29yZGluZyB0byBldmlkZW5jZTogSW1wbGljYXRpb25zIGZvciBjb25zcGlyYXRvcmlhbCwgbW9yYWwsIHBhcmFub3JtYWwsIHBvbGl0aWNhbCwgcmVsaWdpb3VzLCBhbmQgc2NpZW5jZSBiZWxpZWZzLiBKdWRnbWVudCBhbmQgRGVjaXNpb24gTWFraW5nLCAxNSg0KSwgNDc2LgoKLSBSb3RlbGxvLCBDLiBNLiwgS2VsbHksIEwuIEouLCBIZWl0LCBFLiwgVmF6aXJlLCBTLiwgJiBWdWwsIEUuICgyMDE4KS4gVGhlIFNoYXBlIG9mIFJPQyBDdXJ2ZXMgaW4gU2hvb3RlciBUYXNrczogSW1wbGljYXRpb25zIGZvciBCZXN0IFByYWN0aWNlcyBpbiBBbmFseXNpcy4gQ29sbGFicmE6IFBzeWNob2xvZ3ksIDQoMSkuCgotIFRyw6ltb2xpw6hyZSwgQi4sICYgRGplcmlvdWF0LCBILiAoMjAyMCkuIERvbuKAmXQgeW91IHNlZSB0aGF0IGl0cyBjb2xkISBFeHBsb3JpbmcgdGhlIHJvbGVzIG9mIGNvZ25pdGl2ZSByZWZsZWN0aW9uLCBjbGltYXRlIHNjaWVuY2UgbGl0ZXJhY3ksIGlsbHVzaW9uIG9mIGtub3dsZWRnZSwgYW5kIHBvbGl0aWNhbCBvcmllbnRhdGlvbiBpbiBjbGltYXRlIGNoYW5nZSBza2VwdGljaXNtLgoKLSBXaWNraGFtLCBILiAoMjAxNCkuIFRpZHkgZGF0YS4gSm91cm5hbCBvZiBTdGF0aXN0aWNhbCBTb2Z0d2FyZSwgNTkoMTApLCAxLTIzLg==